smartpipe-cli 1.4.0rc3__tar.gz → 1.4.0rc4__tar.gz

{smartpipe_cli-1.4.0rc3 → smartpipe_cli-1.4.0rc4}/CHANGELOG.md

@@ -53,6 +53,15 @@ The identity release.
   power path.
 
 ### The item — one law for everything in a pipe
+- **Nested field paths, everywhere you read.** `where 'user.plan ==
+  "pro"'`, `sort --by metrics.score`, `chart items[0].category`,
+  `summarize 'avg(metrics.score) by user.plan'`, and `"Is {user.plan}
+  worth it?"` in prompts - one grammar (`a.b.c`, `items[0]`,
+  `a.b['weird key']`) across all five surfaces. A column literally
+  named `user.plan` always wins over the traversal, so CSV headers
+  with dots keep working. Extraction names stay flat - paths are for
+  reading, and the error says so.
+
 - **Dates are a type, and they come back ISO.** `{due date}` and
   `{ts datetime}` join the brace grammar. However the model phrases it
   ("Jan 15, 2026", "15/01/2026"), the field canonicalizes to
@@ -84,9 +93,20 @@ The identity release.
   double-underscore namespace that rides every record: `__source` (path plus
   HOW the item was cut - file, lines, jsonl, pages, minutes, segment index),
   `__media` (one transport object for bytes crossing a pipe), `__score`
-  (join), `__invalid`/`__error`/`__raw` (kept failures). Known fields
-  round-trip through saved files; unknown `__` fields warn once and carry.
-  User data owns everything up to one leading underscore.
+  (top_k and join similarity), `__rank` (top_k), `__snapshot` (top_k
+  --stream markers), `__distance` (outliers), `__invalid`/`__error`/`__raw`
+  (kept failures). Known fields round-trip through saved files; unknown `__`
+  fields warn once and carry. User data owns everything up to one leading
+  underscore - which is why the last single-underscore stragglers moved:
+  pre-1.4 `top_k`/`outliers` wrote `_score`/`_rank`/`_snapshot`/`_distance`,
+  and those spellings stay readable for one release (CSV/TSV still sort them
+  into the trailing columns) but are never written again. In the same sweep,
+  `cluster --explode` overwriting an existing `cluster` field became a
+  disclosed overwrite (one stderr warning) instead of a silent one.
+- **top_k ranks can no longer collide.** Ranking keyed on each item's
+  source index, so two page-1 records from different PDFs fought over one
+  slot and a result silently vanished. Ranks are now run-scoped ordinals in
+  arrival order, in both the embedded and precomputed-vector paths.
 - **Records in, records out.** A plain-prompt `map` over records now returns
   a record (`{"result": …}` plus the spine) instead of bare prose - structure
   and provenance survive transformation. Text lines still leave as text.
@@ -114,6 +134,22 @@ The identity release.
   `where`/`summarize` report rows that had no fields; `--strict-rows` turns
   the notes into errors. Multi-line plain output into a pipe warns that
   line framing is ambiguous (use `--output json`).
+- **`.sem` files are strict about their rows now.** A saved pipeline is a
+  program, and a stray plain line in a record stream is a bug, not a
+  shrug: `.sem` runs default to `--strict-rows` and fail at ingestion
+  naming the exact line, before any spend. Opt out with
+  `strict-rows = false` (top-level or per-stage); ad-hoc terminal runs
+  keep the permissive census. Eleven verbs that accepted `--strict-rows`
+  without actually enforcing it now do, and a text-only `where` predicate
+  over a log no longer trips strict mode over "missing fields". A new
+  docs page explains the granularity ladder (file → split → lines/rows)
+  and when mixing happens.
+- **Synthesized records tell you what they summarize.** join pairs carry
+  `__sources` (both parents' refs); reduce windows stamp
+  `{"as": "window", "span": [1, 100], "count": 100}`, groups
+  `{"as": "group", "group": …, "count": N}`; cluster and diff summary
+  rows carry `{"as": "cluster"|"diff", "count": N}`. An audit trail for
+  every row that speaks for other rows - and `--bare` strips all of it.
 
 ### Judgment you can gate, retry, and keep
 - **`--keep-invalid`** (map/extend): after the repair retry fails, keep the
@@ -123,6 +159,13 @@ The identity release.
   `--check FILE` validates rows (exit 0/1), `--example` prints a validating
   instance, bare stdin mode is a REPL. **`--dry-run`** (map/extend) prints
   the composed first request and exits before any model resolution.
+- **`schema --check` stopped failing your own pipelines.** The check is
+  open-world now: rows from the documented extend workflow (original
+  fields kept, spine aboard) pass instead of tripping "additional
+  properties are not allowed" on every row, and `?`-marked fields may be
+  absent, not just null. Extras earn one dim hint; `--strict` restores
+  the closed world for contract enforcement. The schema sent to models
+  at extraction time is unchanged.
 - **Nullability is declared.** `?` on any type (`string?`, `number[]?`, bare
   `field?`) compiles to a null union; bare fields mean scalar-or-scalar-list
   and never admit null (D48, shipped earlier in this cycle).
@@ -171,6 +214,19 @@ The identity release.
   over alphabetical luck (cloud tags compete equally), and speaks in the
   same color voice as the other commands with a menu of detected local
   tags and provider-prefixed cloud examples.
+- **Exit 141 is now deliberate, and the -13 flake is dead.** smartpipe used
+  to arm the default SIGPIPE handler, so a downstream `| head` killed it by
+  raw signal - and, one run in a few hundred, a stray EPIPE on asyncio's
+  own self-pipe killed a healthy run mid-teardown, skipping the receipt
+  (the CI flake that delayed rc3). SIGPIPE now stays ignored; a closed
+  stdout surfaces as BrokenPipeError and exits a silent, deliberate 141
+  with every finally honored. (click 8.4's own EPIPE-to-exit-1 trap is
+  defused on the way.) One visible delta: `kill -PIPE` no longer
+  terminates smartpipe; pipeline behavior is unchanged.
+- **A files-then-stdin wait has a name.** Chaining positional files into a
+  stdin read used to end in a silent hang when nothing was piped. All three
+  chain paths now say once, on stderr:
+  `note: files done - now reading stdin (pipe data or close it; files-only: add < /dev/null)`.
 - **Docs bug, live-caught: `cluster | chart` drew flat bars.** The summary
   rows are one-per-cluster, so charting them counts every theme once. The
   README quick-tour step, the `cluster` help epilog, and the `.sem`
@@ -239,15 +295,24 @@ The identity release.
 
 ### Documents parse where they enter, vectors stay honest
 - **The `ocr-model` role: a document parser at ingestion.** Set
-  `smartpipe config ocr-model mistral/mistral-ocr-latest` (or
-  `SMARTPIPE_OCR_MODEL`, or `--ocr-model` per run) and ingested PDFs and
-  images parse through it - one item per page on the `pages` cut,
-  markdown as the text, disclosed per row and metered as a paid
-  conversion. Mistral refs ride the dedicated `/v1/ocr` wire
-  (live-verified, whole PDFs in one call); any other model ref works as
-  chat-vision with extract-the-text framing, so `ollama/llava` is a free
-  local OCR. Parse failures fall back to the local extraction ladder,
-  disclosed, never fatal. Unset = exactly the old behavior.
+  `ocr-model = "mistral/mistral-ocr-latest"` in config.toml (the setup
+  flow's OCR stage stamps it; `SMARTPIPE_OCR_MODEL` and `--ocr-model`
+  override per run) and ingested PDFs and images parse through it - one
+  item per page on the `pages` cut, markdown as the text, disclosed per
+  row and metered as a paid conversion. Mistral refs ride the dedicated
+  `/v1/ocr` wire (live-verified, whole PDFs in one call); any other
+  model ref works as chat-vision with extract-the-text framing, so
+  `ollama/llava` is a free local OCR. Parse failures fall back to the
+  local extraction ladder, disclosed, never fatal. Unset = exactly the
+  old behavior.
+- **Reader mode does what you configured.** `smartpipe scan.pdf` used to
+  be pinned "zero model calls" even when you had set an ocr-model - the
+  reader now honors the role with the same per-row disclosure, local
+  fallback, and `--max-calls` belt as the verbs (plus a preflight note
+  when more than 20 files will parse). No ocr-model configured = the
+  same free path as always, pinned by a zero-HTTP-calls test. Found en
+  route: the mistral OCR wire was never counted by `--max-calls`
+  anywhere - it is belted now.
 - **The `media-embed-model` role: one joint space for pixels and prose.**
   Text keeps `embed-model`; media routes to the joint-space model
   (jina-clip-v2 and friends). If a run would mix two vector spaces, the
@@ -321,7 +386,26 @@ The identity release.
   overwrites one you chose yourself), and one y/N question adds a backup
   `fallback-model` for provider outages. After a `doctor --probe`,
   catalog entries carry capability chips (`sees, hears - probed 2d ago`)
-  - probed facts, never claims. All the setter subcommands still work.
+  - probed facts, never claims.
+- **`smartpipe use` - setup is one word now.** `use gemini` (or
+  `use claude-opus-4-8`, `use ollama/qwen3:8b`) stamps a coherent bundle
+  in one shot: the model, its paired embedder, and the captions posture,
+  each line disclosed with a ✓ and a reason. No credential for the named
+  provider = a clean refusal that names the exact `auth login` command,
+  nothing stamped. Bare `use` opens the interactive flow. `smartpipe
+  using` answers "what am I running?" with every setting, its value, and
+  WHERE it came from (flag, env, config file, default) in one aligned
+  grid. The config.toml you get is signed by its door -
+  `# stamped by: smartpipe use (2026-07-10T06:38Z)` - so a mystery
+  config explains itself. The old setter subcommands (`config model`,
+  `config embed-model`, …) and profiles are gone; an old config with
+  profile keys loads with one warning and cleans itself on the next
+  save.
+- **The no-model dead end offers a hand.** At a real terminal, the
+  no-model error now ends with `run setup now? [Y/n]` - accept, walk the
+  same setup flow, and it closes with "saved - rerun your command".
+  Scripts and CI see byte-identical output to before (the offer is
+  TTY-gated), and the exit code stays honest.
 
 ### Getting it and keeping it current
 - **One-line install on every platform.**

{smartpipe_cli-1.4.0rc3 → smartpipe_cli-1.4.0rc4}/CITATION.cff

@@ -7,5 +7,5 @@ authors:
     given-names: Prabal
 repository-code: "https://github.com/prabal-rje/smartpipe"
 license: Apache-2.0
-version: 1.4.0rc3
+version: 1.4.0rc4
 date-released: '2026-07-08'

{smartpipe_cli-1.4.0rc3 → smartpipe_cli-1.4.0rc4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: smartpipe-cli
-Version: 1.4.0rc3
+Version: 1.4.0rc4
 Summary: Semantic pipes and queries for your terminal - PDFs, images, audio, video, and text through Unix verbs that understand.
 Project-URL: Homepage, https://github.com/prabal-rje/smartpipe
 Project-URL: Changelog, https://github.com/prabal-rje/smartpipe/blob/main/CHANGELOG.md
@@ -49,7 +49,7 @@ Description-Content-Type: text/markdown
 Run PDFs, images, audio, video, and text through Unix verbs that understand
 their input.
 
-[![smartpipe demo - click to play with sound and controls](https://github.com/prabal-rje/smartpipe/releases/download/demo-assets/smartpipe-teaser-v4.gif)](https://prabal-rje.github.io/smartpipe/demo/)
+[![smartpipe demo - click to play with sound and controls](https://github.com/prabal-rje/smartpipe/releases/download/demo-assets/smartpipe-teaser-v5.gif)](https://prabal-rje.github.io/smartpipe/demo/)
 
 *[▶ Watch the full 86-second demo](https://prabal-rje.github.io/smartpipe/demo/) - sound, controls, no download.*
 
@@ -93,7 +93,7 @@ and runs that tool's own upgrade.
 smartpipe auth login
 ```
 
-No ChatGPT plan? Use local [Ollama][ollama] or a cloud API key - see
+No ChatGPT plan? Bring a cloud API key, or run local [Ollama][ollama] - see
 [Models & providers][models].
 
 ## Examples
@@ -165,7 +165,7 @@ calls by construction.
 
 ```bash
 # 1. point smartpipe at a model (ChatGPT login, a cloud key, or local Ollama)
-smartpipe config
+smartpipe use
 
 # 2. ask one question across a folder of mixed documents
 smartpipe map "What does this say about pricing?" 'docs/*.pdf'
@@ -277,7 +277,7 @@ Full docs: **[prabal-rje.github.io/smartpipe][docs]**.
 - [Install][install] - packages and platforms
 - [Working with files & media][files] - PDFs, scans, images, audio, video
 - [CLI reference][cli] - every flag, format, and exit code
-- [Models & providers][models] - local Ollama, cloud keys, ChatGPT login
+- [Models & providers][models] - ChatGPT login, cloud keys, local Ollama
 - [Privacy & security][privacy]
 
 ## How to cite

{smartpipe_cli-1.4.0rc3 → smartpipe_cli-1.4.0rc4}/README.md

@@ -11,7 +11,7 @@
 Run PDFs, images, audio, video, and text through Unix verbs that understand
 their input.
 
-[![smartpipe demo - click to play with sound and controls](https://github.com/prabal-rje/smartpipe/releases/download/demo-assets/smartpipe-teaser-v4.gif)](https://prabal-rje.github.io/smartpipe/demo/)
+[![smartpipe demo - click to play with sound and controls](https://github.com/prabal-rje/smartpipe/releases/download/demo-assets/smartpipe-teaser-v5.gif)](https://prabal-rje.github.io/smartpipe/demo/)
 
 *[▶ Watch the full 86-second demo](https://prabal-rje.github.io/smartpipe/demo/) - sound, controls, no download.*
 
@@ -55,7 +55,7 @@ and runs that tool's own upgrade.
 smartpipe auth login
 ```
 
-No ChatGPT plan? Use local [Ollama][ollama] or a cloud API key - see
+No ChatGPT plan? Bring a cloud API key, or run local [Ollama][ollama] - see
 [Models & providers][models].
 
 ## Examples
@@ -127,7 +127,7 @@ calls by construction.
 
 ```bash
 # 1. point smartpipe at a model (ChatGPT login, a cloud key, or local Ollama)
-smartpipe config
+smartpipe use
 
 # 2. ask one question across a folder of mixed documents
 smartpipe map "What does this say about pricing?" 'docs/*.pdf'
@@ -239,7 +239,7 @@ Full docs: **[prabal-rje.github.io/smartpipe][docs]**.
 - [Install][install] - packages and platforms
 - [Working with files & media][files] - PDFs, scans, images, audio, video
 - [CLI reference][cli] - every flag, format, and exit code
-- [Models & providers][models] - local Ollama, cloud keys, ChatGPT login
+- [Models & providers][models] - ChatGPT login, cloud keys, local Ollama
 - [Privacy & security][privacy]
 
 ## How to cite

{smartpipe_cli-1.4.0rc3 → smartpipe_cli-1.4.0rc4}/demo/video/src/scenes/ScaleB.tsx

@@ -15,10 +15,10 @@ const LINES: readonly TerminalLine[] = [
   },
   { kind: "ord", at: sec(3.2), text: "#1" },
   { kind: "field", at: sec(3.3), k: "text", v: "Checkout 502 bug …" },
-  { kind: "field", at: sec(3.4), k: "_score", v: "0.93" },
+  { kind: "field", at: sec(3.4), k: "__score", v: "0.93" },
   { kind: "ord", at: sec(3.9), text: "#2", gap: true },
   { kind: "field", at: sec(4.0), k: "text", v: "Pay page hangs …" },
-  { kind: "field", at: sec(4.1), k: "_score", v: "0.88" },
+  { kind: "field", at: sec(4.1), k: "__score", v: "0.88" },
 ];
 
 /** Scene 6 — scale B: search the index by meaning. */

{smartpipe_cli-1.4.0rc3 → smartpipe_cli-1.4.0rc4}/docs/comparison.md

@@ -57,8 +57,8 @@ Some tools fit certain jobs better:
 
 - **Local model probing, with explicit cloud setup** - if no chat model is configured,
   smartpipe probes local Ollama first. If it finds no usable model, it stops with
-  setup instructions. Cloud calls happen only after you choose a cloud model,
-  profile, or login path.
+  setup instructions. Cloud calls happen only after you choose a cloud model or
+  login path.
 
 - **`grep --color=auto`-style adaptive output** - human view at a terminal, JSONL when piped, with no flag.
 

{smartpipe_cli-1.4.0rc3 → smartpipe_cli-1.4.0rc4}/docs/concepts/feeding-smartpipe.md

@@ -13,8 +13,10 @@ live in [the item](the-item.md); this page is the ingestion mechanics.
 | ONE binary document (`< report.pdf`) | one whole-document item, spooled and extracted |
 | an image / audio / video stream | one media item, bytes carried to the model |
 
-Mixed streams are legal; `--strict-rows` (or `SMARTPIPE_STRICT_ROWS=1`) makes
-them an error when your pipeline demands one kind.
+Mixed streams are legal in an interactive pipe; `--strict-rows` (or
+`SMARTPIPE_STRICT_ROWS=1`) makes them an error naming the first mixed row,
+and `.sem` runs are strict by default (unattended = loud) - the full story is
+[the granularity ladder](granularity.md).
 
 ## Named files
 
@@ -85,8 +87,14 @@ ingestion made visible, and the front half of the
 ```bash
 smartpipe report.pdf | head -1
 smartpipe notes.txt --as lines | smartpipe write 'copy/{name}'
+# {name} = the source file's name, carried by __source - notes.txt becomes copy/notes.txt
 ```
 
+The reader is zero-calls by default. A configured `ocr-model` is the one
+exception: PDFs and images parse through it exactly as the ingesting verbs
+would (disclosed per row; `--max-calls` caps it, and a folder of 20+
+parseable files gets a preflight note before the first call).
+
 ## What rides along
 
 Every item carries its provenance in the `__source` spine field (path, cut

smartpipe_cli-1.4.0rc4/docs/concepts/granularity.md

@@ -0,0 +1,109 @@
+# The granularity ladder
+
+Every pipe starts with one decision: **what is one item?** This page is that
+decision, top to bottom - when each cut applies, how streams end up mixed,
+and what the census and `strict-rows` do about it.
+
+## The ladder
+
+From coarsest to finest:
+
+| Rung | One item is | You get it from |
+|---|---|---|
+| **file** | a whole file (its extracted text + media) | naming a path, `--as file`, redirecting one document (`< report.pdf`) |
+| **split** | a piece of a file: a page, N tokens, N minutes/seconds | `smartpipe split --by pages/tokens/minutes/seconds` |
+| **lines / rows** | one text line, one JSONL record, one CSV row | piping text in, `.jsonl`/`.csv` paths, `--as lines/jsonl/csv` |
+
+Three rules pick the rung when you don't say:
+
+1. **Paths decide the default.** `.jsonl`/`.ndjson` cut into strict records,
+   `.csv`/`.tsv` into header-named rows; every other path is one whole-file
+   item.
+2. **stdin sniffs per line.** A line that parses as a JSON object is a
+   record; anything else is a text line. One redirected binary document is a
+   single whole-file item.
+3. **`--as` overrides everything.** `--as file` slurps (even stdin);
+   `--as lines` keeps every line text, even lines that look like JSON;
+   `--as jsonl` demands one record per line, loudly; `--as csv` reads the
+   first line as the header.
+
+`split` sits between file and lines: it cuts *below* the file into items
+that remember how they were cut (each chunk carries a `__source` spine, so
+`write` can reassemble them in order at the far end).
+
+## When each cut applies
+
+- **file** - the unit of meaning is the document: summarize a report,
+  caption an image, transcribe a call.
+- **split** - one document is too big, or you want the pieces addressable
+  ("page 3 says..."). Media clips cut by minutes/seconds; documents by pages
+  or tokens.
+- **lines/rows** - the unit of meaning is the row: logs, exports, JSONL
+  datasets. This is where per-item verbs earn their keep.
+
+## How mixing happens
+
+Auto mode sniffs stdin per line, so a stream can come out **mixed** - some
+records, some plain text. It happens innocently:
+
+- a log file where some lines are JSON events and some are free text,
+- `cat a.jsonl b.txt` gluing two kinds together,
+- a producer that writes headers, blank-ish banners, or progress lines
+  between records.
+
+Mixing is legal in an interactive pipe. But a verb given both kinds treats
+them differently (a record has fields; a text line only has `text`), so
+silence would lie. Hence the census.
+
+## The census note
+
+At end of stream, a mixed pipe gets exactly one stderr note:
+
+```
+note: input: 812 records · 3 plain lines
+```
+
+Nothing fails; the note tells you the stream wasn't what you probably
+thought. Declare a kind to make it go away: `--as jsonl` (records) or
+`--as lines` (text).
+
+## strict-rows
+
+When your pipeline *demands* one kind, make mixing an error:
+
+- `--strict-rows` on the verb, or
+- `SMARTPIPE_STRICT_ROWS=1` in the environment.
+
+Strict mode fails at the **first** mixed row, naming it - before the verb
+sees it, before any model call could spend on it:
+
+```
+error: input: line 4 is a plain text line in a record stream
+  --strict-rows demands one kind - declare it: --as jsonl (records) or --as lines (text).
+```
+
+Strictness also covers field misses in the free verbs: `where` errors when
+rows have no fields for a field predicate to read, `summarize` when rows
+lack the `by` field.
+
+### `.sem` runs are strict by default
+
+A script runs unattended, and unattended means loud: `smartpipe run x.sem`
+behaves as if `--strict-rows` were set, for single-stage files and every
+pipeline stage alike. Mixed-by-design pipelines opt out in the file:
+
+```toml
+verb = "split"
+strict-rows = false
+```
+
+(per stage or top-level in pipeline files; an explicit CLI flag or
+environment variable still wins). Interactive pipes keep the permissive
+census note. The details live in
+[.sem files - strict rows by default](../reference/sem-files.md#strict-rows-by-default).
+
+## See also
+
+- [The item](the-item.md) - the five laws behind all of this
+- [Feeding smartpipe](feeding-smartpipe.md) - the ingestion mechanics table
+- [split](../verbs/split.md) - cutting below the file

{smartpipe_cli-1.4.0rc3 → smartpipe_cli-1.4.0rc4}/docs/concepts/models-and-providers.md

@@ -3,18 +3,18 @@
 ## What's a "model"?
 
 A model is the AI that reads your instruction and produces the answer. smartpipe
-doesn't contain one - it sends your text to a model running either **locally** (on
-your machine, via Ollama) or in the **cloud** (OpenAI, Anthropic, or any
-compatible service). You choose which; smartpipe just talks to it.
+doesn't contain one - it sends your text to a model running either in the
+**cloud** (OpenAI, Anthropic, or any compatible service) or **locally** (on
+your machine, via Ollama). You choose which; smartpipe just talks to it.
 
-## Local vs. cloud
+## Cloud vs. local
 
-| | Local (Ollama) | Cloud |
+| | Cloud | Local (Ollama) |
 |---|---|---|
-| Cost | Free | Small charge per use |
-| Privacy | Nothing leaves your machine when Ollama is local | Text and supported media go to the provider |
-| Setup | Install Ollama, pull a model | Get an API key |
-| Speed / quality | Depends on your hardware | Usually faster and stronger |
+| Cost | Small charge per use | Free |
+| Privacy | Text and supported media go to the provider | Nothing leaves your machine when Ollama is local |
+| Setup | Log in with ChatGPT, or get an API key | Install Ollama, pull a model |
+| Speed / quality | Usually faster and stronger | Depends on your hardware |
 
 With no configured chat model, smartpipe tries local Ollama first. If it finds a
 non-embedding model, it uses that model and prints the exact `ollama/...` name.
@@ -23,7 +23,7 @@ If no model is configured and no usable Ollama model is found, smartpipe stops w
 a setup screen. It does not silently fall through to a cloud provider just because
 a key or login exists.
 
-Once you choose a cloud model, profile, or ChatGPT login path, the relevant item data
+Once you choose a cloud model or ChatGPT login path, the relevant item data
 for that run goes to that provider. That is an explicit provider choice, not a hidden
 fallback.
 
@@ -142,40 +142,39 @@ Example: the table floors Gemini at 128k, but the probe can discover that
 `SMARTPIPE_CONTEXT_TOKENS=32000` overrides everything. If an estimate is still wrong,
 `reduce` self-corrects: a rejected chunk is split in half and retried.
 
-## Profiles: named setups you can switch between (D30)
+## One command, one coherent setup: `smartpipe use`
 
-A profile bundles the existing config keys (model, embed-model, concurrency,
-output) under a name. Three ship built in:
-
-| Profile | Chat | Embeddings | For |
-|---|---|---|---|
-| `openai` | gpt-5.4-mini | text-embedding-3-small | cloud preset |
-| `gemini` | gemini-3.1-flash-lite | gemini/gemini-embedding-001 | cloud multimodal preset |
-| `local` | ollama/gemma-4-e2b | embeddinggemma | local preset when Ollama runs on your machine |
+`smartpipe use` stamps a COMPLETE bundle - a chat model plus the embedder
+that pairs with it - in one command:
 
 ```bash
-smartpipe config profile              # list (the active one marked)
-smartpipe config profile local        # switch
-SMARTPIPE_PROFILE=gemini smartpipe map …  # one-off, no file change (D24: env is the override)
+smartpipe use                    # interactive: text model, embeddings, OCR
+smartpipe use gemini             # gemini-3.1-flash-lite + gemini-embedding-001
+smartpipe use openai             # gpt-5.4-mini + text-embedding-3-small
+smartpipe use ollama             # the best installed local model + a local embedder
+smartpipe use gpt-5.4-mini       # any model ref: that model + its provider's pairing
 ```
 
-The cloud presets are **multimodal by default**: they set
-`allow-captions = true`, so images and audio convert to text through the
-profile's own model when a run needs it (fractions of a cent each, every
-conversion disclosed per row). Picking the profile is the consent; the wizard
-states this. Bare no-profile setups keep the conservative `--allow-captions`
-opt-in.
+A provider whose key is absent refuses with the `smartpipe auth login` fix
+and stamps nothing - never a partial setup. Re-running `use` refreshes the
+whole bundle, which is also the cure for a drifted config.
+
+Cloud picks are **multimodal by default**: they stamp `allow-captions = true`,
+so images and audio convert to text through your own model when a run needs
+it (fractions of a cent each, every conversion disclosed per row). Picking
+the cloud setup is the consent; the command says so. Local setups keep the
+conservative `--allow-captions` opt-in.
 
-Create your own as a `[profiles.NAME]` table in the config file (keys: model,
-embed-model, concurrency, output, allow-captions). Direct keys beat the active
-profile (a direct set is the most recent intent); flags and env vars beat
-both. Profiles never hold API keys.
+Every save also writes a receipt at the top of `config.toml` - `# stamped
+by: smartpipe use (…)` - so the file documents how it got that way. (Profiles
+from older versions are retired: their keys load with one warning, are
+ignored, and disappear on the next save.)
 
 ## Setting a default
 
 ```bash
-smartpipe config model ollama/qwen3:8b     # save a default
-smartpipe config show                       # see the effective settings + where each comes from
+smartpipe use gpt-5.4-mini                  # save a default (any provider: smartpipe use ollama works the same)
+smartpipe using                             # see the effective settings + where each comes from
 ```
 
 Override the default for a single command with `--model`:
@@ -193,16 +192,13 @@ When the same setting is specified more than one way, the most specific wins:
 --model flag  >  SMARTPIPE_MODEL env var  >  config file  >  Ollama autodetect
 ```
 
-`smartpipe config show` prints each value with its origin, so you can see which value takes effect.
+`smartpipe using` prints each value with its origin, so you can see which value takes effect.
 
 ## Two models: chat and embedding
 
 Most verbs use a **chat** model. `embed` and `top_k` use a separate
-**embedding** model, configured independently:
-
-```bash
-smartpipe config embed-model nomic-embed-text
-```
+**embedding** model, configured independently - the embeddings stage of
+`smartpipe use` sets it (or edit `embed-model` in `config.toml` directly).
 
 ## See also
 
@@ -212,7 +208,8 @@ smartpipe config embed-model nomic-embed-text
 
 ## The stt-model role
 
-`smartpipe config stt-model openai/whisper-1` names a dedicated remote
+The `stt-model` config key (e.g. `stt-model = "openai/whisper-1"` in
+`config.toml`) names a dedicated remote
 transcriber. When set, it runs FIRST in the audio ladder (a configured
 transcriber signals wanting verbatim text - LLM hearing paraphrases),
 falling back to the LLM rung and local `whisper` on failure. It is a paid
@@ -233,7 +230,7 @@ account. Only the `openai` wire exists today; the key accepts
 
 ## The ocr-model role
 
-`smartpipe config ocr-model mistral-ocr-latest` names a dedicated document
+The `ocr-model` config key - the OCR stage of `smartpipe use` sets it - names a dedicated document
 parser. When set, ingested PDFs and images run through it instead of the
 local extraction ladder - page markdown becomes the item text, and a
 multi-page PDF becomes one item per page (the `__source` spine carries the
@@ -249,7 +246,7 @@ The role is provider-agnostic:
 - A **mistral** ref rides the dedicated `/v1/ocr` wire (needs
   `MISTRAL_API_KEY`; the endpoint charges per page).
 - **Any other ref** reads pages through the normal chat-vision wire with an
-  extract-the-text framing - `smartpipe config ocr-model ollama/llava` is a
+  extract-the-text framing - `ocr-model = "ollama/llava"` is a
   free local OCR. For PDFs on this rung, pages with a healthy text layer
   keep their locally extracted text (zero calls); only thin, image-bearing
   pages (scans) go through the model.
@@ -262,7 +259,8 @@ ladder with a note - never a hard stop. `--ocr-model` (per run) >
 
 ## The media-embed-model role
 
-`smartpipe config media-embed-model jina/jina-clip-v2` names a JOINT
+The `media-embed-model` config key (e.g. `media-embed-model =
+"jina/jina-clip-v2"` in `config.toml`) names a JOINT
 text+image embedder. When set, `embed` and `top_k` route media items to it
 as pixels while text items keep using `embed-model` - and a text query can
 rank an image corpus, because both live in the same space.

{smartpipe_cli-1.4.0rc3 → smartpipe_cli-1.4.0rc4}/docs/concepts/output-formats.md

@@ -76,9 +76,15 @@ The rules that make it a real table:
 - **CSV is RFC 4180** (proper quoting, CRLF line endings); **TSV** uses tabs, and
   replaces any tab or newline inside a value with a space (a tab inside a TSV cell
   would corrupt the columns).
-- **`top_k` scores land last.** A `_score` column sorts to the right of your data,
+- **`top_k` scores land last.** A `__score` column sorts to the right of your data,
   where a spreadsheet reader expects it.
 
+> **Migrating from pre-1.4:** `top_k` and `outliers` used to write `_score`
+> (and `_rank`/`_snapshot`/`_distance`) with a single underscore. Those stamps
+> now live in the reserved `__` namespace - update your `jq`/`sort --by`
+> references to `__score` etc. The old spellings are still read for one
+> release (they keep sorting last in CSV/TSV), but are no longer written.
+
 CSV and TSV need **structured** output - there have to be named columns. Ask for
 fields with braces or `--schema`; a plain-text prompt with `--output csv` is a
 usage error that tells you so.