reflex-skills 0.4.1 → 0.5.0

package/README.md

@@ -35,7 +35,7 @@ alias reflex="node $PWD/apps/cli/dist/reflex.js"
 | Command | What it does |
 | --- | --- |
 | `doctor` | Node and SQLite check, detected history locations and file counts, install targets, agent CLIs |
-| `scan --days <n>` | Ask per source, then read and store redacted workflow traces. `--source <claude-code\|codex\|cursor>` (repeatable), `--claude-dir`, `--codex-dir`, `--cursor-dir`, `--yes`, `--full` (re-read every file instead of reusing unchanged ones) |
+| `scan --days <n>` | Ask per source, then read and store redacted workflow traces. `--source <claude-code\|codex\|cursor>` (repeatable), `--claude-dir`, `--codex-dir`, `--cursor-dir`, `--yes`, `--full` (re-read every file instead of reusing unchanged ones), `--embed-minutes <n>` (most time spent comparing new sessions by meaning; default 5, the rest continue next scan) |
 | `sessions` | Sessions from the last scan, redacted |
 | `suggest` | Detect repeated workflows and review them interactively (`--no-interactive`, `--all`) |
 | `show <n\|name>` | A suggestion in detail: steps, support, uncertainty |
@@ -53,9 +53,11 @@ alias reflex="node $PWD/apps/cli/dist/reflex.js"
 | `run <request>` | Start Claude Code or Codex on the model the router picks for this kind of task. `--dry-run` shows the pick and why, `--tool`, `--model`, `--yes`. Asks first unless autopilot is on |
 | `route` | The router: picks per kind of task, settings, randomized trials and bench results. `autopilot on\|off`, `explore <percent>\|off [--protect <kind...>]` |
 | `bench <task>` | Run one task on several Claude Code models, each in a fresh copy of the current repository, scored by `--check "<command>"`. `--models`, `--setup`, `--runs`, `--timeout`, `--keep`, `--yes` |
+| `improve [skill]` | For skills you already have: lists them with the sessions that loaded each; with a name, compares the skill with those sessions and proposes an edit (asks before reading the skill and before changing it; keeps a backup). `--file`, `--yes` |
+| `agents` | Sessions that launched agents (Cursor): how many, which kinds, and how many tool calls the agents made |
 | `stats` | Counts from local data |
 | `reset` | Delete Reflex's own data (not installed skills or the model) |
-| `model [status\|fetch\|remove]` | Optional local embedding model for matching requests by meaning (`fetch` downloads 23 MB) |
+| `model [status\|fetch\|use\|remove]` | Optional local embedding model for matching requests by meaning. `fetch` downloads bge-m3 (multilingual, 587 MB); `fetch --light` the English-only all-MiniLM-L6-v2 (23 MB); `use <name>` switches between installed ones |
 
 Prompts default to No. Without a terminal, nothing is read or written unless you pass `--yes`.
 
@@ -90,8 +92,12 @@ as inspect CI, install dependencies, patch, run tests, inspect the diff), normal
 file categories and outcome. Copies, resumed sessions and quick retries count once. Traces are
 compared on intent (0.40), action sequence (0.30), commands and tools (0.15), file categories
 (0.10) and repository (0.05), then clustered. Intent compares the requests' words (stemmed terms,
-phrases and a small concept lexicon); with the optional model (`reflex model fetch`) it also compares
-their meaning, so "make the login page load faster" and "the sign-in screen is really slow" match. A cluster becomes a suggestion only if it has at
+phrases and a small concept lexicon) in any language: Japanese, Chinese and Thai are split into
+words, Korean particles are stripped (배포를 → 배포), filler words are dropped (published lists plus
+everyday fillers, `casual-stopwords.ts`) and words your own history uses across unrelated kinds of
+work are quieted, more so as the history grows; with the optional model (`reflex model fetch`) it also compares
+their meaning, so "make the login page load faster", "the sign-in screen is really slow" and
+"로그인 화면이 너무 느려" match. A cluster becomes a suggestion only if it has at
 least 3 independent sessions with consistent behavior; one-offs, read-and-answer sessions and
 generic "read, edit, test" patterns are not suggested.
 
@@ -101,6 +107,72 @@ follow-up messages ("don't change the CI workflow") become constraints when the
 appears in at least 2 sessions; only the words those sessions share are kept. No transcript excerpts.
 Details: [docs/architecture.md](docs/architecture.md).
 
+## Matching by meaning: which model
+
+Matching by words always runs. A local embedding model adds matching by meaning; it is optional and
+runs on your machine (WebAssembly, no native code, no network after the download).
+
+| | `reflex model fetch` (default) | `reflex model fetch --light` |
+| --- | --- | --- |
+| Model | bge-m3, int8 | all-MiniLM-L6-v2, int8 |
+| Reads | 100+ languages, and requests that mix them | English (Latin script); other text is matched by words only |
+| Download | 587 MB | 23 MB |
+| Memory while embedding | about 2.8 GB | about 0.6 GB |
+| Time per session, 1 / 4 threads | about 2.9 s / 0.8 s | about 0.09 s / 0.03 s |
+
+Measured on Apple silicon with real-length requests; older CPUs can be up to about 3x slower. The
+`Embedding benchmark` workflow (`.github/workflows/embedding-bench.yml`; on pull requests that change embedding or matching code, by hand, and weekly)
+measures load time, time per request and peak memory for both models on Linux (x64 and ARM),
+Windows and macOS (Intel and Apple silicon), and checks matching quality is the same everywhere;
+locally: `REFLEX_PERF=bge-m3 REFLEX_TEST_MODELS=~/.reflex/models pnpm vitest run packages/analysis/test/eval/perf.test.ts`. On
+English requests MiniLM matches at least as well as bge-m3, so English-only users lose nothing with
+`--light`. Embedding uses up to 4 cores (`REFLEX_EMBED_THREADS` to change). Before downloading,
+`model fetch` estimates the first scan's time from your stored sessions and recommends `--light`
+on machines with under 8 GB of memory or fewer than 4 cores. Each scan spends at most
+`--embed-minutes` (default 5) embedding, newest sessions first, and the next scan continues; a
+machine without memory for the model skips it with a warning. Vectors are cached per model, so
+`reflex model use <name>` switches without re-embedding what either model already did.
+
+## Skills you already have
+
+If most sessions of a repeated workflow already loaded skills you wrote, Reflex does not suggest a
+new skill for it; `reflex scan` says which of your skills handle it instead. Only skills specific to
+that workflow count (loaded at least twice as often there as in your other sessions), so general
+guidance you load everywhere never hides a suggestion. Skill folders that are symlinks into a
+skills repo are followed: Reflex reads and edits the real `SKILL.md`, so changes show up in that
+repo's `git status`. `--dry-run` shows the review, the proposal and the check without writing.
+
+`reflex improve <skill>` compares one of your skills with the sessions that loaded it (at least 2).
+The rule is that a skill keeps producing the same thing; improving it means getting there faster
+or cheaper.
+
+- **What it never touches.** Your text is never edited: Reflex only appends a marked "Learned from
+  your sessions" section (replaced on the next run, never duplicated). Sections whose heading names
+  the output (`Output format`, `Response`, `Template`, `Report`, …) and anything between
+  `<!-- reflex:fixed:start -->` and `<!-- reflex:fixed:end -->` are treated as the output contract:
+  the appended section says to keep them exactly, and nothing proposed may write a summary or
+  correct the format.
+- **Proposed by default: faster.** Commands that worked in those sessions for steps your skill
+  already names, so the agent doesn't work them out again.
+- **Shown, not written: cheaper.** If a cheaper model was about as reliable on those sessions,
+  `reflex improve` says so; `reflex run` uses it. The skill file isn't changed for that.
+- **Opt-in with `--behavior`: changes what the agent does.** Steps most sessions took that the
+  skill doesn't mention, and corrections you repeated. Steps the skill names that rarely happened
+  are reported; Reflex never removes a step.
+- **Checked before it's applied.** `--verify "<a task this skill handles>"` runs that task with the
+  current and the proposed skill, each in a fresh copy of the current repository (non-persistent
+  Claude Code sessions on your account), and compares the shape of the two answers: headings,
+  tables, lists, labels, JSON keys. Values such as times and names may differ; the format may not.
+  If it changed, the edit is not applied (`--force` overrides). Answers are compared in memory and
+  never saved.
+- **Recorded.** Every proposal is attached to the skill in Reflex's database: fingerprints before
+  and after, what was proposed, the check result, whether you applied it and where the backup is.
+  `reflex improve <skill> --history` lists them. Your file changes only if you confirm; the previous
+  version goes to `~/.reflex/backups`.
+
+A session counts as using a skill when the agent called it (Claude Code's `Skill` tool) or read
+its `SKILL.md`.
+
 ## Measuring a skill
 
 Each suggestion shows what the workflow costs today: median tokens and tool calls per run. When you
@@ -156,8 +228,9 @@ Cursor isn't routed: it records no tokens to compare, and its editor takes no mo
 
 - Transcripts are read locally and treated as data, never as instructions.
 - Reflex makes no network connections; outgoing sockets are blocked in-process. The one exception is
-  `reflex model fetch`, which you run explicitly to download the optional embedding model (pinned
-  files, verified by SHA-256). It reads no transcripts and uploads nothing; scans stay offline.
+  `reflex model fetch`, which you run explicitly to download an optional embedding model (pinned
+  files, streamed to disk and verified by SHA-256). It reads no transcripts and uploads nothing;
+  scans stay offline.
 - Secrets, emails, home paths and your user name are redacted before anything is stored or shown.
 - Scans are incremental: a transcript that has not changed since the last scan is not opened again.
   Redacted traces are kept after their transcript is deleted (by you, or by the agent's own cleanup),
@@ -176,8 +249,21 @@ Threat model and mitigations: [docs/security.md](docs/security.md).
 | Codex | `$CODEX_HOME/sessions/YYYY/MM/DD/rollout-*.jsonl` | `~/.codex/skills/<name>/SKILL.md` | Parser follows the Codex protocol types and is tested on fixtures; not yet run against real rollouts or a live Codex session |
 | Cursor | `~/.cursor/projects/*/agent-transcripts/*/*.jsonl` | `~/.cursor/skills/<name>/SKILL.md` | Parser tested on fixtures and on real agent transcripts. Cursor doesn't record tool results, so pass/fail outcomes aren't available for its sessions |
 
-Override locations with `REFLEX_CLAUDE_DIR`, `REFLEX_CODEX_DIR`, `REFLEX_CURSOR_DIR`,
-`REFLEX_CLAUDE_SKILLS_DIR`, `REFLEX_CODEX_SKILLS_DIR`, `REFLEX_CURSOR_SKILLS_DIR` and `REFLEX_HOME`. `REFLEX_DEBUG=1` prints local counters (counts only).
+**Agents a session launched.** When a Cursor chat launches agents (its `Task` / `Subagent` tool),
+Reflex records each one (kind, model, background or not, its short description) and reads the
+agents' transcripts (`<chat>/subagents/*.jsonl`). Their commands, files and tool calls count toward
+the chat's workflow; the message each agent was given is written by the parent agent, so it is
+never treated as your request. `reflex agents` lists the chats that launched the most, and
+`reflex sessions` notes the count. Claude Code's agents are not read yet.
+
+**Conductor.** Conductor runs its own Claude Code and Codex, which save sessions to `~/.claude` and
+`~/.codex` like any other session, so Reflex reads them with those sources. A session whose folder is
+a Conductor workspace (`~/conductor/workspaces/<repo>/<workspace>`) counts toward `<repo>`, not
+toward each workspace, and `reflex sessions` marks it "via Conductor". `reflex doctor` shows whether
+the workspace folder exists. Cursor and OpenCode sessions run through Conductor are not read yet.
+
+Override locations with `REFLEX_CLAUDE_DIR`, `REFLEX_CODEX_DIR`, `REFLEX_CURSOR_DIR`, `REFLEX_CONDUCTOR_DIR`,
+`REFLEX_CLAUDE_SKILLS_DIR`, `REFLEX_CODEX_SKILLS_DIR`, `REFLEX_CURSOR_SKILLS_DIR` and `REFLEX_HOME`. `REFLEX_EMBED_THREADS` sets the embedding threads. `REFLEX_DEBUG=1` prints local counters (counts only).
 
 ## Walkthrough
 
@@ -201,11 +287,13 @@ by the tests; it is not part of the published CLI.
 ## Limitations
 
 - Confidence is a ranking heuristic, not a probability.
-- Without the model, intent matching uses stemmed TF-IDF plus a small built-in lexicon. Requests worded
-  with no words in common need `reflex model fetch` to match.
-- The embedding model (all-MiniLM-L6-v2) is English-only and does not know developer jargon as well as
-  the lexicon ("GitHub Actions is red" vs. "CI is failing"), so it is combined with word matching,
-  never used alone. Clusters found only by meaning may get a generic name.
+- Without the model, intent matching uses stemmed TF-IDF plus a small built-in lexicon (English).
+  Requests worded with no words in common, or asked in different languages, need
+  `reflex model fetch` to match.
+- Embedding models do not know developer jargon as well as the lexicon ("GitHub Actions is red" vs.
+  "CI is failing"), so they are combined with word matching, never used alone. With the light
+  model, requests in other scripts (Korean, Japanese, ...) are matched by words only. Clusters
+  found only by meaning may get a generic name.
 - Each source reads at most its 2,000 most recent files per scan.
 - Sponsor integrations (Alike, Darwin) are not included: no public API documentation was available
   to build against, and the product works without them.