@uehaj/sys1grep 0.5.0-next.0 → 0.5.0-next.1

package/README.md

@@ -14,7 +14,7 @@ and applies a threshold.
 ./sys1grep -n -e "customer is angry or frustrated" tickets.txt
 ```
 
-[![sys1grep demo: a Japanese meaning finds refund requests in six languages; "asking for a refund" vs "about a refund"; -Q finds the answer](docs/demo.svg)](https://uehaj.github.io/sys1grep/)
+[![sys1grep demo: a Japanese meaning finds refund requests in six languages; "asking for a refund" vs "about a refund"; -Q finds the answer](docs/demo.svg)](https://uehaj.github.io/jev-semgrep/)
 
 <sub>▶ Click the demo, or open <a href="https://uehaj.github.io/sys1grep/">uehaj.github.io/sys1grep</a>, for the full demo on the landing page.</sub>
 
@@ -134,8 +134,7 @@ as a plain regex, with no request at all. It prefilters its AND term: only the l
 ever ask that term's meanings, so a cheap regex in front of a meaning cuts both the bill and the wait. A line is sent
 only if some term's regexes all hold for it (a term with no regex holds for every line), so with
 `-e '/re/' -a A -e B` a line without `re` is still sent, asked `B` only. A query of regex terms alone
-sends nothing, except that `--sentence` (`=jev`, the default) still asks Jev where wrapped lines
-break; use `--sentence=rules` to stay offline. Anything that isn't shaped like `/…/flags` is still a meaning, so
+sends nothing, even with `--unit=sentence-by-jev`: the rules alone then decide where wrapped lines join. Anything that isn't shaped like `/…/flags` is still a meaning, so
 `-e '/etc 以下のファイルを変更している'` (no closing `/`) is unaffected; a meaning that really starts
 and ends with `/` can be written with a leading space to dodge the regex reading.
 
@@ -172,24 +171,49 @@ $ ./sys1grep -o -n -e '/[A-Z]+-\d+/' -a 'the ticket is still open' notes.txt   #
 Every line sent to Jev costs money and time, so the cheapest line is the one never sent. From the widest cut
 to the narrowest:
 
-- **Which files.** `-r` skips `.git`, `node_modules`, binary files, likely secrets and what git ignores;
-  [`git sys1grep`](#as-a-git-subcommand-git-sys1grep) searches tracked files only. `--include` / `--exclude`
-  (file-name globs) and `--changed-within` (`30m`, `7d`, `today`, `this-week`, a date) narrow them further.
-  A meaning that restricts itself to a language or a time of change narrows them by itself: see
-  [Scope from the meaning](#scope-from-the-meaning).
+- **Which files.** `-r` skips `.git`, `node_modules`, binary files, likely secrets, generated files (source maps,
+  minified JS/CSS, lock files) and what git ignores; [`git sys1grep`](#as-a-git-subcommand-git-sys1grep) searches
+  tracked files only. `--include` / `--exclude` (file-name globs) and `--changed-within` (`30m`, `7d`, `today`,
+  `this-week`, a date) narrow them further. A meaning that restricts itself to a language or a time of change
+  narrows them by itself: see [Scope from the meaning](#scope-from-the-meaning).
 - **Which lines.** A [regex term](#regex-terms) is matched locally, and only the lines it holds for are asked
-  its AND term's meanings. Blank lines are never sent.
+  its AND term's meanings. Blank lines are never sent. Only the first `-M`/`--max-columns` characters of a line
+  (default 2000, 8000 with `-z`) are sent: it is still searched and judged, just not past that cutoff.
 - **How many times.** [`--dedup`](#one-line-per-template---dedup) judges one line per template: lines that
   differ only in ids, numbers, times or paths share one answer.
-- **Check before paying.** `--dry-run` sends nothing and prints the files, how many lines each would send and
-  every request with its questions. Its last line estimates the input tokens and, for TypeSafe itself, the price
-  (`~3178 input tokens, ~$0.000133`; within about 10%). `-i` shows the same totals on the terminal and sends only
-  after `y`.
+- **Check before paying.** `--dry-run` sends nothing and prints the settings the search would run with, the
+  files, how many lines each would send and every request with its questions. Its last line estimates the input
+  tokens and, for TypeSafe itself, the price (`~3178 input tokens, ~$0.000133`; within about 10%). `-i` shows the
+  same totals on the terminal and sends only after `y`.
+  Before the bulk of requests, every target is sized (`--max-filesize`, default 10M); one over it is skipped
+  outright, like `rg`'s own `--max-filesize`, named on stderr (`-y` does not affect it). A `.gz` is sized
+  decompressed (zlib stops at the limit, so an oversized one is never inflated in full). stdin is sized once it
+  is read (it is already in memory), and skipped the same way if it is over; a `--cached` or `<tree>:` target
+  is a blob rather than a file, sized from its content (already read in one `git cat-file --batch` call, not a
+  fresh git process per blob); `-g`'s commits stay out of this, since each is already bounded by `-M` when it
+  is sent. The input about to be sent is priced too (`--max-cost`,
+  default 1 USD); over it, one question asks to continue on the terminal, `-y` answers it yes, and without a
+  terminal it is exit 2 (this applies with `-q` too — scripts pass `-y`).
 
 ```sh
 $ sys1grep --dry-run -r --include='*.log' --changed-within=today -e '/ERROR|FATAL/' -a 'a customer is affected' logs/
 ```
 
+`--verbose` (or `--dry-run`) also shows *where* a setting that was not typed on the command line came from
+(`SYS1GREP_OPTS`, an environment variable, `~/.config/sys1grep/.env`, or a preset's default), so a result that
+surprises you can be traced back to its source. The key's value never appears, only which option or variable
+supplied it:
+
+```sh
+$ SYS1GREP_OPTS='--level strict' sys1grep --verbose -e "the API key is read from a file" .
+sys1grep: endpoint api.typesafe.ai/v1/systemone (default), model jev-latest (default)
+sys1grep: key: SYS1GREP_API_KEY (~/.config/sys1grep/.env)
+sys1grep: SYS1GREP_OPTS: --level strict
+sys1grep: options: --level strict (SYS1GREP_OPTS) = -t 0.7 -T 0.3, --chunk 30, -j 8, scope on, -M 2000, --max-filesize 10M, --max-cost 1
+sys1grep: file ./a.py: 120 lines, 120 to send
+…
+```
+
 ## Install
 
 Two ways to use it: as a command-line tool (this section), or as a Claude Code skill
@@ -381,11 +405,18 @@ tests/tickets/sub/b.txt
 ```
 
 `-r` walks directories in sorted order and skips `.git`, `node_modules`, `.ssh`, `.aws`, `.gnupg`, `.kube`,
-`.docker`, binary files (a NUL byte in the first 8 KB, or a PDF; UTF-16 with a BOM is read as text) and files
+`.docker`, binary files (a NUL byte in the first 8 KB, or a PDF; UTF-16 with a BOM is read as text), files
 that usually hold secrets (`.env*`, `.netrc`, `.npmrc`, `.pypirc`, `.pgpass`, `.git-credentials`, `*.pem`,
-`*.key`, `*.p12`, `*.pfx`, `*.jks`, `*.keystore`, `id_rsa*` and friends; names compared without case). **Every line that is searched is sent to the TypeSafe API**, so point `-r` at a directory
+`*.key`, `*.p12`, `*.pfx`, `*.jks`, `*.keystore`, `id_rsa*` and friends; names compared without case) and
+generated files (`*.map`, `*.min.js`, `*.min.css`, `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`,
+`Cargo.lock`, `poetry.lock`, `composer.lock`, `Gemfile.lock`, `go.sum`). **Every line that is searched is sent
+to the TypeSafe API**, so point `-r` at a directory
 you mean to scan. Inside a git repository, `-r` also skips what git ignores (`.gitignore`, `.git/info/exclude`, the
 global excludes file), so build output and local files stay home; a tracked file is searched even if it matches.
+A file named `*.gz` (a rotated log: `app.log.1.gz`) is read decompressed, as `zgrep` does, and printed by its
+name on disk (`app.log.1.gz:12:...`); `--include='*.gz'` picks those, the binary sniff sees the decompressed
+bytes, a gzipped secret (`private.key.gz`) is skipped like the plain one, and a corrupt `.gz` is an unreadable file.
+Only gzip, only by the name.
 A file named explicitly on the command line is always searched, even if it matches the skip list or is ignored;
 so is a directory that git ignores, when you name it (`sys1grep -r -e ... dist`). `-l` prints each
 matching file once, in the order matches are found, and works with or without `-r`. `-c` prints the number of
@@ -453,6 +484,25 @@ applies even to tracked files. `--include`, `--exclude` and `--changed-within` f
 by a pathspec included. `--changed-within` reads the working tree's modification times, not git history: right after
 a clone or a checkout, every file it wrote counts as just changed. For help use `git sys1grep -h`: git takes `--help` itself and looks for a man page.
 
+`--cached`, `--untracked` and a `<tree>...` choose what is searched instead of the working tree, as `git grep` has
+them (only one of the three at a time):
+
+```sh
+$ git sys1grep --cached -e "a retry is attempted"       # staged, uncommitted changes included
+$ git sys1grep --untracked -e "a retry is attempted"    # tracked files plus untracked ones (.gitignore still applies)
+$ git sys1grep -n -e "a retry is attempted" main v0.3.1 -- '*.py'
+main:src/job.py:42:    retry(job, times=3)
+v0.3.1:src/job.py:40:    retry(job)
+```
+
+A `<tree>` (a branch, tag, commit or `@{u}`) is any argument before `--` that resolves as a revision; several may
+be given, searched in the order given, each line prefixed with the name as typed (`@{u}:path`, not the branch it
+resolves to). `--changed-within` needs the working tree: a blob (`--cached`, a `<tree>`) has no mtime of its own.
+A `<tree>`'s own pathspec takes a glob too, same as elsewhere in sys1grep (`git diff-tree` against the empty
+tree, not `git ls-tree`'s own literal/directory-prefix match); `--include` / `--exclude` (by name) still work.
+**An old `<tree>` can hold a secret an ordinary file once carried and was later removed from**: the skip list
+drops files by name only, not by what changed since.
+
 ### Everything that is *not* something
 
 ```sh
@@ -480,15 +530,15 @@ Matching records are printed NUL-terminated too, so pipe them through `tr '\0' '
 File names (`-l`) and counts (`-c`) stay on newlines, as they do in grep. With `-z`, `-n` numbers records,
 `-A` / `-B` / `-C` count neighbouring records, and `--chunk` counts records per request.
 
-### One sentence at a time (`--sentence`)
+### One sentence at a time (`--unit=sentence-by-*`)
 
-`--sentence` judges each sentence instead of each line. The output is still lines, as in grep: every line a
+`--unit=sentence-by-jev` (or `sentence-by-rule`, below) judges each sentence instead of each line. The output is still lines, as in grep: every line a
 matching sentence touches is printed, and on a terminal the sentence itself is in bold yellow (a regex match inside it, in grep's bold red).
 Wrapped lines are joined before splitting, so a sentence that runs over several lines is judged as one.
 [`tests/prose.txt`](tests/prose.txt) wraps an English paragraph and a Japanese one:
 
 ```sh
-$ ./sys1grep -n --sentence -e "the author admits they made a mistake" tests/prose.txt
+$ ./sys1grep -n --unit=sentence-by-jev -e "the author admits they made a mistake" tests/prose.txt
 1:I should have checked the input
 2:before shipping, and that was my
 3:mistake. Next time I will add a test
@@ -500,14 +550,14 @@ The sentence starts on line 1 and ends at `mistake.` on line 3; only that part i
 On a terminal, with both meanings and `-C 3` for context, the colors show where each sentence starts and ends
 inside a line: lines 3 and 9 are colored only up to the end of the matching sentence, and lines 4-7 are context (`-`):
 
-![--sentence -C 3 --color: the matching sentences in bold yellow, up to mistake. on line 3 and 返金してほしいです。 on line 9; lines 4 to 7 as context](docs/sentence.svg)
+![--unit=sentence-by-jev -C 3 --color: the matching sentences in bold yellow, up to mistake. on line 3 and 返金してほしいです。 on line 9; lines 4 to 7 as context](docs/sentence.svg)
 
 `-o` prints only the matching sentences, one per line, as `grep -o` prints only the matching part.
 `-n` then gives the line where the sentence starts. Japanese is joined without a space, as are Chinese,
 Thai, Lao, Khmer, Myanmar and Tibetan, which do not put spaces between words:
 
 ```sh
-$ ./sys1grep -n -o --sentence -e "the author admits they made a mistake" -e "customer is asking for a refund" tests/prose.txt
+$ ./sys1grep -n -o --unit=sentence-by-jev -e "the author admits they made a mistake" -e "customer is asking for a refund" tests/prose.txt
 1:I should have checked the input before shipping, and that was my mistake.
 8:先週買った掃除機が初日から動かないので返金してほしいです。
 ```
@@ -515,20 +565,20 @@ $ ./sys1grep -n -o --sentence -e "the author admits they made a mistake" -e "cus
 Without `-o`, `-c` counts lines and `-A` / `-B` / `-C` count lines, as usual. With `-o` they count sentences.
 With `-z`, each record is split on its own and matching records are printed whole.
 
-Jev finds a matching sentence inside a long line on its own, so `--sentence` is not needed for accuracy.
+Jev finds a matching sentence inside a long line on its own, so `--unit=sentence-by-jev` is not needed for accuracy.
 Use it to see which sentence matched, to get the sentences with `-o`, and when AND should hold within one
 sentence: the expression is evaluated per sentence. For the same reason `-v X` alone prints every line
-with at least one sentence that is not X; to find lines that are not X as a whole, leave `--sentence` off.
+with at least one sentence that is not X; to find lines that are not X as a whole, leave `--unit` at `line`.
 
 Japanese and Chinese entries often end without `。`: a chat message, a support ticket, a memo line. Joining
-them would glue separate entries into one "sentence". So by default (`--sentence`, the same as
-`--sentence=jev`) sys1grep asks Jev about each unpunctuated break next to a script written without word
+them would glue separate entries into one "sentence". So with
+`--unit=sentence-by-jev` sys1grep asks Jev about each unpunctuated break next to a script written without word
 spaces: "does this line break end a sentence or entry, or is it a wrap inside a sentence?" It sends 30 lines
 per request with one yes/no per break, and keeps the lines apart when the answer is 0.7 or more. On
 [`tests/corpus.txt`](tests/corpus.txt) this keeps the four one-line Japanese tickets apart, so
-`--sentence` finds the same refund requests (lines 14 and 18) as a line-by-line search, where the rules alone
+`--unit=sentence-by-jev` finds the same refund requests (lines 14 and 18) as a line-by-line search, where the rules alone
 merged the tickets and missed line 18. The extra requests cost about as much as one more meaning; use
-`--sentence=rules` to skip them. Breaks between English lines are never asked: joining them keeps a space,
+`--unit=sentence-by-rule` to skip them. Breaks between English lines are never asked: joining them keeps a space,
 and the full stop still ends the sentence.
 
 Where a newline cannot be inside a sentence, lines are not joined: at a blank line, next to brackets or
@@ -538,6 +588,38 @@ own. Sentences are cut by `Intl.Segmenter` ([Unicode UAX #29](https://unicode.or
 which splits at `.` `!` `?` `。` `！` `？` but also after abbreviations such as `Mr.`. Logs are not prose:
 consecutive log lines that start with a letter, such as `WARN ...` after `ERROR ...`, get joined.
 
+### One function at a time (`--unit=function`)
+
+In code the question is usually about a function ("retries on network failure"), and one line of it rarely
+says so. `--unit=function` judges each function and prints its lines, with `-n` giving file line numbers:
+
+```sh
+$ ./sys1grep -n --unit=function -e "retries on network failure" src/net.js
+src/net.js:3:async function fetchWithRetry(url) {
+src/net.js:4:  for (let i = 0; i < 5; i++) {
+src/net.js:5:    try { return await fetch(url); }
+src/net.js:6:    catch { await sleep(2 ** i * 100); }
+src/net.js:7:  }
+src/net.js:8:}
+```
+
+A function runs from a funcname line to the line before the next one, as `git grep -W` finds it; lines
+before the first funcname line are one unit, and blank lines at a function's end are not printed. The
+funcname lines are git's: the `diff=<driver>` attribute in `.gitattributes`, then `diff.<driver>.xfuncname`
+in git config. Without one, sys1grep has its own rule for JavaScript/TypeScript (a top-level `function`,
+`class`, or `const` / `let` / `var` bound to a function) and Python (`def` / `class`, nested ones too); a decorator line (`@retry`) starts
+the function it decorates. Otherwise it uses git's default: a line starting with a letter, `_` or `$`. git's own builtin drivers
+(`diff=python` and so on) are not read, so a driver without an `xfuncname` falls back the same way:
+
+```sh
+$ printf '*.go diff=golang\n' >> .gitattributes
+$ git config diff.golang.xfuncname '^(func|type)[[:space:]]'
+```
+
+`-M` defaults to 8000 characters here, as with `-z`; a longer function is judged on its first 8000.
+`--chunk` counts functions, `-A`/`-B`/`-C` and `-c` count lines, as with the sentence units. `-z`, `-g` and
+`-o` are refused.
+
 ### One line per template (`--dedup`)
 
 Cost is proportional to the text sent, and machine-generated logs are mostly one skeleton with a different
@@ -561,7 +643,7 @@ could change a match, and those kinds are kept apart (one of the two requests ab
 Measured on real logs, with nothing kept: a 43,071-line system log folds into 550 templates (1.2% of the
 bytes), `install.log` to 21.1%, a Claude Code transcript (jsonl) only to 56.8%. It is for machine-generated
 logs; prose has no shared skeleton, and a meaning that reads a timestamp folds almost nothing. With `-z` or
-`--sentence` the records or sentences fold instead of lines.
+`--unit=sentence-by-*` the records or sentences fold instead of lines.
 
 After a search, the summary line says what the folding saved: `(2 folded by --dedup, ~235 input tokens /
 ~$0.000010 saved, 21%)` above. It estimates what the folded lines would have cost as requests of their own and subtracts
@@ -589,15 +671,44 @@ The expensive model reads only what Jev kept. Asking why `./.env` is no longer r
 `git log` (168 commits), Claude's input fell from 22,059 tokens to 1,356, the total cost with Jev's from
 $0.094 to $0.013, with the same answer (#69). It pays when the answer sits in a few lines.
 
-- The matching lines leave the machine a second time, to Anthropic.
+- The matching lines leave the machine a second time, to Anthropic (or whatever TOOL talks to).
 - `-n`, `-A/-B/-C`, `-p` and file names go in as they would print; colors never do. No match runs nothing (exit 1).
-- `SYS1GREP_SUMMARIZER` picks the TOOL of a bare `--summarize` (only `claude` so far), `SYS1GREP_SUMMARIZER_MODEL` its model.
+- `SYS1GREP_SUMMARIZER` picks the TOOL of a bare `--summarize`, `SYS1GREP_SUMMARIZER_MODEL` its model.
 - `-q`, `-l` and `-c` print no lines, so they cannot be combined with it.
+- The answer is plain text: an LLM that is not told writes Markdown, noise on a terminal, so plain is asked for.
+  `--summarize-format=markdown` or `=html` asks for those instead (`… --summarize-format=html … > summary.html`).
+  The answer prints as it comes, unchecked. In `SYS1GREP_OPTS` it is a standing preference, ignored without `--summarize`.
 - With `--dedup`, the TOOL gets what Jev got: each template's representative once, marked `(×N like it)`,
   not every line its answer was reused for.
 - More than 200 KB (about 50k tokens) is not sent at all: exit 2, with the size, before the TOOL is paid.
   It is never cut short, since a summary of the first part would read as a summary of all of it.
 
+`--summarize-prompt=TEXT` adds your own instruction after the fixed one (how long, what to focus on):
+
+```sh
+$ sys1grep -r -n --summarize --summarize-prompt="3 lines or fewer, just which file to fix" \
+    -e "the API key is read from a file" .
+```
+
+It needs `--summarize`; empty TEXT is the same as leaving it out. It comes after the `--summarize-format` sentence,
+so TEXT can override the format.
+
+Other TOOLs:
+
+```sh
+$ sys1grep --summarize=llm -e "..." FILE            # Simon Willison's llm, tools off (no -T given)
+$ sys1grep --summarize=pi -e "..." FILE             # pi --print --no-tools --no-session ...
+$ SYS1GREP_SUMMARIZER_MODEL=qwen3.5:9b sys1grep --summarize=ollama -e "..." FILE
+$ SYS1GREP_SUMMARIZER_MODEL=some-id sys1grep --summarize=lmstudio -e "..." FILE   # model id from GET /v1/models
+$ SYS1GREP_SUMMARIZER_MODEL=some-id sys1grep --summarize=http://localhost:8080/v1 -e "..." FILE  # llama.cpp, vLLM, LocalAI, a gateway
+```
+
+`ollama` and `lmstudio` talk to a local OpenAI-compatible server (`POST /v1/chat/completions`) by `fetch`, no
+CLI: the matching lines never leave the machine a second time. A plain `http(s)://` URL is any other
+OpenAI-compatible server. All three need `SYS1GREP_SUMMARIZER_MODEL`: none has a default model.
+`SYS1GREP_SUMMARIZER_API_KEY` goes as `Authorization: Bearer` to a URL TOOL only (never `SYS1GREP_API_KEY`,
+which is Jev's). `OLLAMA_HOST` moves ollama's host, as it does for the `ollama` CLI itself.
+
 ## Use it from Claude Code
 
 There is a Claude Code skill that runs sys1grep for you: describe what you are looking for in plain words
@@ -651,7 +762,11 @@ usage: sys1grep [OPTION]... -e MEANING|-Q QUESTION [-a MEANING] [-v MEANING]...
   -T THRESH    negative threshold: "not X" when probability < THRESH (overrides --level)
                with -t 0.6 -T 0.3 a line at 0.3..0.6 matches neither X nor not-X
   -r           recurse into directories (current directory when FILE is omitted);
-               skips .git, node_modules, binary files, likely secrets and what git ignores
+               skips .git, node_modules, binary files, likely secrets, generated files and what git ignores
+  --cached     git sys1grep only: search the index instead of the working tree, as git grep --cached
+  --untracked  git sys1grep only: also search untracked files (.gitignore still applies), as git grep --untracked
+  <tree>...    git sys1grep only: a branch, tag, commit or @{u} before -- searches that revision's tree
+               instead, as git grep does; output is prefixed <tree>: with the name as typed
   --include=GLOB, --exclude=GLOB  with -r and git sys1grep, only files whose name matches GLOB, or not
                (with -r a file named on the command line is always searched; git sys1grep's pathspecs are filtered)
   --changed-within=WHEN  with -r and git sys1grep, only files modified within 30m / 2h / 7d / 2w, since a date
@@ -670,13 +785,26 @@ usage: sys1grep [OPTION]... -e MEANING|-Q QUESTION [-a MEANING] [-v MEANING]...
                on ambiguous lines, not just speed
   -j N         concurrent requests (default 8)
   -n           print line numbers
-  -z, --null-data  judge NUL-terminated records instead of lines, and print them NUL-terminated (see "Records that span several lines" above)
-  --sentence[=HOW] judge each sentence instead of each line; HOW is jev (default) or rules (see "One sentence at a time" above)
-  -o           with --sentence, print only the matching sentences
+  --unit=UNIT  the unit of judgement: line (default), zero, sentence-by-jev, sentence-by-rule or function
+  -z, --null-data  --unit=zero: judge NUL-terminated records instead of lines, and print them NUL-terminated (see "Records that span several lines" above);
+               with --unit=sentence-by-*, each record is split into sentences
+  --unit=sentence-by-jev|sentence-by-rule  judge each sentence instead of each line (see "One sentence at a time" above)
+  --unit=function  judge each function and print its lines (see "One function at a time" above)
+  -o           with --unit=sentence-by-*, print only the matching sentences
   -p           print each meaning's probability at the end of the line
-  --dry-run    send nothing; print the endpoint, each file searched and each request with its questions
+  --dry-run    send nothing; print the settings the search would run with (and their source, when not the
+               command line), each file searched and each request with its questions
   --verbose    print the same to stderr while searching
   -i, --interactive  show what --dry-run would send, and search only after y on the terminal
+  -M NUM, --max-columns=NUM  send at most the first NUM characters of a line (or record, with -z); still
+               searched and judged, just not past that cutoff (default 2000, 8000 with -z)
+  --max-filesize=SIZE  size every target first (K/M/G, default 10M); over it, skip it outright, like rg's own
+               --max-filesize (-y does not affect it). stdin is sized once read, and skipped the same way if
+               it is over; a --cached / <tree>: target is a blob, sized from its content; -g's commits stay
+               out of this (each is already bounded by -M when sent)
+  --max-cost=USD  price the input about to be sent; over it (default 1), ask to continue on the terminal; -y
+               answers yes without asking; no terminal and the limit exceeded is exit 2, unchanged by -q
+               (scripts pass -y); -i already asks unconditionally and earlier, so this does not ask again
   --dedup      judge one line per template and reuse its answer for the rest (see "One line per template" above)
   --color[=WHEN] auto (default: color when stdout is a terminal) / always / never; bare --color means auto
                file and line number use grep's colors; with -p, probabilities are
@@ -766,14 +894,14 @@ fmt -w 100000 essay.txt | sys1grep -n -e "the author admits they made a mistake"
 
 Each output line is then a whole paragraph, and `-n` counts the lines of `fmt`'s output, not of the file.
 `fmt` puts a space where it joins two Japanese lines; Jev reads through it. To judge sentences across
-wrapped lines you need neither: `--sentence` already joins them.
+wrapped lines you need neither: `--unit=sentence-by-jev` already joins them.
 
 ### How do I search a JSONL chat log one message at a time?
 
 Take the text out with `jq` and end each message with NUL, then judge records with `-z`:
 
 ```sh
-jq -j '.content + "\u0000"' chat.jsonl | sys1grep -z -n --sentence -e "the customer is asking for a refund" | tr '\0' '\n'
+jq -j '.content + "\u0000"' chat.jsonl | sys1grep -z -n --unit=sentence-by-jev -e "the customer is asking for a refund" | tr '\0' '\n'
 ```
 
 Adjust `.content` to where your log keeps the text. Each message becomes one record, so a sentence never

package/docs/sentence.svg

@@ -1,6 +1,6 @@
 <svg xmlns="http://www.w3.org/2000/svg" width="808" height="240" font-family="SFMono-Regular,Menlo,Consolas,monospace" font-size="13">
 <rect width="100%" height="100%" rx="6" fill="#1e1e2e"/>
-<text x="16" y="29" xml:space="preserve"><tspan fill="#cdd6f4">$ sys1grep -n --sentence -C 3 --color \</tspan></text>
+<text x="16" y="29" xml:space="preserve"><tspan fill="#cdd6f4">$ sys1grep -n --unit=sentence-by-jev -C 3 --color \</tspan></text>
 <text x="16" y="49" xml:space="preserve"><tspan fill="#cdd6f4">    -e "the author admits they made a mistake" -e "customer is asking for a refund" tests/prose.txt</tspan></text>
 <text x="16" y="69" xml:space="preserve"><tspan fill="#8ce99a">1</tspan><tspan fill="#66d9e8">:</tspan><tspan fill="#ffd43b" font-weight="700">I should have checked the input</tspan></text>
 <text x="16" y="89" xml:space="preserve"><tspan fill="#8ce99a">2</tspan><tspan fill="#66d9e8">:</tspan><tspan fill="#ffd43b" font-weight="700">before shipping, and that was my</tspan></text>

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@uehaj/sys1grep",
-  "version": "0.5.0-next.0",
+  "version": "0.5.0-next.1",
   "description": "grep by meaning, across languages. TypeSafe Jev scores every line against a meaning; combine meanings with AND/OR/NOT.",
   "keywords": [
     "grep",