eklavya 1.14.0 → 1.15.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. package/dist/assets/tutor/SKILL.md +180 -0
  2. package/dist/assets/tutor/references/focus-and-level.md +156 -0
  3. package/dist/assets/tutor/references/grading.md +113 -0
  4. package/dist/assets/tutor/references/writing-mcq.md +133 -0
  5. package/dist/cli.js +66 -17
  6. package/dist/cli.js.map +1 -1
  7. package/dist/eval/extract-json.js +81 -0
  8. package/dist/eval/extract-json.js.map +1 -0
  9. package/dist/eval/extraction-score.js +115 -0
  10. package/dist/eval/extraction-score.js.map +1 -0
  11. package/dist/eval/history-stats.js +230 -0
  12. package/dist/eval/history-stats.js.map +1 -0
  13. package/dist/eval/question-checks.js +220 -0
  14. package/dist/eval/question-checks.js.map +1 -0
  15. package/dist/hooks/checkpoint-quiz.js +1 -1
  16. package/dist/hooks/lib.js +38 -13
  17. package/dist/hooks/lib.js.map +1 -1
  18. package/dist/hooks/prompt-submit-nudge.js +155 -0
  19. package/dist/hooks/prompt-submit-nudge.js.map +1 -0
  20. package/dist/hooks/session-start.js +32 -14
  21. package/dist/hooks/session-start.js.map +1 -1
  22. package/dist/hooks/stop-quiz-check.js +4 -3
  23. package/dist/hooks/stop-quiz-check.js.map +1 -1
  24. package/dist/plugin/.claude-plugin/plugin.json +1 -1
  25. package/dist/plugin/agents/tutor.md +19 -5
  26. package/dist/plugin/hooks/CLAUDE.md +83 -2
  27. package/dist/plugin/hooks/hooks.json +12 -0
  28. package/dist/plugin/hooks/run.mjs +3 -3
  29. package/dist/plugin/skills/CLAUDE.md +107 -15
  30. package/dist/plugin/skills/setup/SKILL.md +1 -1
  31. package/dist/plugin/skills/tutor/SKILL.md +119 -304
  32. package/dist/plugin/skills/tutor/references/focus-and-level.md +156 -0
  33. package/dist/plugin/skills/tutor/references/grading.md +113 -0
  34. package/dist/plugin/skills/tutor/references/writing-mcq.md +133 -0
  35. package/dist/slug.js +63 -0
  36. package/dist/slug.js.map +1 -1
  37. package/dist/stdin.js +131 -0
  38. package/dist/stdin.js.map +1 -0
  39. package/dist/store.js +16 -2
  40. package/dist/store.js.map +1 -1
  41. package/dist/tools/get_session_quiz_plan.js +1 -1
  42. package/dist/tools/get_session_quiz_plan.js.map +1 -1
  43. package/dist/tools/record_attempt.js +15 -3
  44. package/dist/tools/record_attempt.js.map +1 -1
  45. package/package.json +1 -1
  46. package/dist/assets/tutor-skill.md +0 -365
@@ -30,15 +30,29 @@ The main session asks quiz questions as multiple choice through the `AskUserQues
30
30
  ```
31
31
  Which of these does `httpOnly` actually prevent?
32
32
 
33
- A) JavaScript reading the cookie via document.cookie
34
- B) The cookie being sent to a different origin
35
- C) The cookie surviving a browser restart
33
+ A) The cookie being sent to a different origin
34
+ B) The cookie surviving a browser restart
35
+ C) JavaScript reading the cookie via document.cookie
36
36
  D) The cookie being read over plain HTTP
37
37
 
38
38
  Reply with a letter, or "teach me" if you'd rather I explain it.
39
39
  ```
40
40
 
41
- Same rules as the skill's *Multiple choice* section: four options, three plausible distractors, the stem alone in `record_attempt`'s `question`, the labels in `options`, `format: "mcq"`, and the grade capped at 4. The only thing that changes is who draws the box.
41
+ The answer is at C there because the plan said `answer_position: 3`, and
42
+ **`answer_position` matters more here than anywhere else**: in the main session
43
+ a tool draws the options, and here you letter them yourself, so nothing but you
44
+ enforces the slot. Left to instinct the right answer lands at A nearly every
45
+ time, and a learner needs only a handful of questions to notice that and start
46
+ replying "A" without reading.
47
+
48
+ "teach me" is this renderer's version of the tool's "Other" choice. Treat it as
49
+ a blank, not a decline: grade 0 with `outcome: "dont_know"`, then teach.
50
+
51
+ Same rules as the tutor skill's `skills/tutor/references/writing-mcq.md`: four
52
+ options, three plausible distractors, the correct one at `answer_position`, the
53
+ stem alone in `record_attempt`'s `question`, the labels in `options`,
54
+ `format: "mcq"`, and the grade capped at 4. The only thing that changes is who
55
+ draws the box.
42
56
 
43
57
  ## You do not write code
44
58
 
@@ -60,4 +74,4 @@ You are competing for attention with an agent that is producing code. Ask one qu
60
74
 
61
75
  Never record a grade for a question that was not answered. Record a decline (`outcome: "declined"`) only when they actually say so.
62
76
 
63
- "I don't know" is not silence and not a decline — it is a request. Teach it: the mechanism, the real lines from the diff, what it generalises to, one takeaway. Then record grade 0 with `outcome: "dont_know"` and move on. See the `tutor` skill for the full shape.
77
+ "I don't know" is not silence and not a decline — it is a request. Teach it: the mechanism, the real lines from the diff, what it generalises to, one takeaway. Then record grade 0 with `outcome: "dont_know"` and move on. The full shape is in the tutor skill's `skills/tutor/references/grading.md`.
@@ -30,11 +30,12 @@ triple in here, even in a comment, fails that test on purpose.
30
30
  `scripts/bump-version.sh` bumps `plugin.json` and `mcp/package.json`, nothing
31
31
  else.
32
32
 
33
- ## The four hooks, out of `hooks.json`
33
+ ## The five hooks, out of `hooks.json`
34
34
 
35
35
  | Event | Matcher | Timeout | Script | Job |
36
36
  |---|---|---|---|---|
37
37
  | SessionStart | — | 10s | `session-start` | stamp `meta.current_session`, print the profile banner and the standing log directive |
38
+ | UserPromptSubmit | — | 10s | `prompt-submit-nudge` | re-state the log directive in one line, but only for a session that has logged nothing after a grace window |
38
39
  | PreToolUse | `Bash` | 10s | `pre-tool-gate` | in `enforced` mode only, deny a `git commit` whose session gate has not passed |
39
40
  | PostToolUse | `mcp__.*log_session_concepts` | 10s | `checkpoint-quiz` | one mid-task question, `interleaved` cadence only |
40
41
  | Stop | — | 15s | `stop-quiz-check` | block the turn and demand a quiz |
@@ -54,10 +55,90 @@ stdin defensively, run the body, exit 0 silently on any throw. A new hook body
54
55
  goes inside `await run(async (input) => { ... })` and returns an exit code; it
55
56
  never calls `process.exit` itself. Helpers follow the same rule —
56
57
  `openExisting()` returns `null` rather than throwing on a missing or corrupt
57
- database, and deliberately does not migrate or seed (four hooks racing a
58
+ database, and deliberately does not migrate or seed (several hooks racing a
58
59
  migration on session start is a corruption story). The one non-zero code is the
59
60
  Stop hook's `return 2`, which is how Stop blocks; the reason goes on stderr.
60
61
 
62
+ ## A hook must never *wait*, either
63
+
64
+ Exiting 0 on a throw covers the loud failure. The quiet one is worse: a hook
65
+ that blocks never errors, never logs, and stalls the session on every tool call
66
+ that triggers it — with nothing for the developer to report except that Claude
67
+ Code got slow.
68
+
69
+ `readInput` used to be `for await (const chunk of process.stdin)`, which has
70
+ exactly one exit: EOF. Ponytail's issue #443 reports Claude Code on Windows
71
+ running a hook through a PowerShell block that swallows the piped JSON, so `end`
72
+ never fires. **Nothing here verifies that mechanism** — there is no Windows
73
+ machine in the loop — so what this defends against is the consequence, a stdin
74
+ that never ends, which the tests reproduce directly. `run.mjs` is careful about
75
+ everything else — Node version, four resolution candidates, a self-expiring heal
76
+ claim, exit 0 on every throw — and this was the one gap.
77
+
78
+ `mcp/src/stdin.ts` closes it, and `eklavya statusline` shares it: both read a
79
+ JSON blob the host pipes in, both must degrade rather than hang, and two copies
80
+ would be one copy getting the fix. Three things matter about it.
81
+
82
+ **The primary bound is on silence, not on total time.** A flat cap truncates a
83
+ payload still arriving when it fires, and truncated JSON does not fail loudly —
84
+ it fails as `{}`, so the hook runs to completion having quietly decided the
85
+ session has no cwd and no id. The idle timer resets on every chunk, so a payload
86
+ is safe as long as it keeps making progress. The total cap behind it *can* still
87
+ truncate, and claiming otherwise would be the same overclaim in the other
88
+ direction — it is a deliberate trade, and `totalMs` is generous against how long
89
+ a real payload takes.
90
+
91
+ **It pauses the stream, not just the listeners.** This is the line the rest of
92
+ it depends on. `setEncoding` puts stdin in flowing mode and a flowing stdin holds
93
+ an active libuv handle, so removing the `data` listener resolves the read and
94
+ leaves the process alive. The hooks hide that — `run()` ends in `process.exit` —
95
+ but `eklavya statusline` just returns, and under exactly the no-EOF condition
96
+ this exists for it printed the dials and then lingered forever, once per status
97
+ bar refresh. `test/stdin.test.ts` spawns the statusline for that reason: it is
98
+ the caller with no `process.exit` behind it, so it is the honest test of whether
99
+ the read lets go.
100
+
101
+ **Every timer is `unref`'d**, so a pending timer adds no latency to a hook that
102
+ has already finished. On its own that does not let the process exit — see above.
103
+
104
+ **There is an `error` handler.** A stream that errors never emits `end`, so
105
+ without one the read waits on something that is not coming. It is also
106
+ load-bearing beyond that: an unhandled `error` on `process.stdin` is an async
107
+ exception `run()`'s `try`/`catch` could not have caught.
108
+
109
+ **It costs something, and the cost is worth stating.** On a host that swallows
110
+ the pipe this turns an infinite hang into `idleMs` per invocation, and PreToolUse
111
+ matches every `Bash` call — so +2s per command until the host is fixed. Two
112
+ seconds a command is bad; a frozen session is worse.
113
+
114
+ `HOOK_STDIN` is 2s idle / 5s total, well under the 10s `hooks.json` grants (15
115
+ for Stop) — a read that outlives its host timeout is a read the developer waits
116
+ on, and `test/stdin.test.ts` asserts the relationship rather than the number.
117
+ `STATUSLINE_STDIN` is 150ms / 250ms, and the total is pinned at or below the
118
+ 250ms flat cap the inline reader had before it: splitting one budget into idle
119
+ plus total made the worst case four times worse for the one caller whose latency
120
+ a human sees, which a test now prevents.
121
+
122
+ That suite spawns a real hook, writes a payload, and **never closes stdin**;
123
+ against the old code all three cases hang until the test kills them.
124
+
125
+ `stripBom` runs before every `JSON.parse` here. Some Windows shells prepend a
126
+ byte-order mark, and `JSON.parse` throws on input that looks perfectly
127
+ well-formed in a terminal and in any editor — another silent nothing-happens.
128
+
129
+ ## `quiet` is not an off switch
130
+
131
+ It suppresses the session-start banner and the status bar — things the developer
132
+ looks at. It does **not** suppress the standing directive, and it does not gate
133
+ the `UserPromptSubmit` nudge, because both are `additionalContext` the model
134
+ reads rather than output anyone sees.
135
+
136
+ That was a bug for a while, and a bad one: `session-start` returned before
137
+ pushing the directive, so a developer who turned the greeting off logged
138
+ nothing, was never quizzed, and saw `mode: ambient` in `get_config` the whole
139
+ time. Two tests encoded it as intended behaviour. `mode: off` is the off switch,
140
+ and it is the only one.
141
+
61
142
  ## The Stop hook blocks in `ambient` too
62
143
 
63
144
  Commonly got wrong. `ambient` is not "never interrupts" — `stop-quiz-check.ts`
@@ -13,6 +13,18 @@
13
13
  ]
14
14
  }
15
15
  ],
16
+ "UserPromptSubmit": [
17
+ {
18
+ "hooks": [
19
+ {
20
+ "type": "command",
21
+ "command": "node",
22
+ "args": ["${CLAUDE_PLUGIN_ROOT}/hooks/run.mjs", "prompt-submit-nudge"],
23
+ "timeout": 10
24
+ }
25
+ ]
26
+ }
27
+ ],
16
28
  "PreToolUse": [
17
29
  {
18
30
  "matcher": "Bash",
@@ -11,8 +11,8 @@
11
11
  *
12
12
  * "command": "node", "args": ["${CLAUDE_PLUGIN_ROOT}/hooks/run.mjs", "<name>"]
13
13
  *
14
- * and every platform-specific decision lives here instead of in four shell
15
- * scripts.
14
+ * and every platform-specific decision lives here instead of one shell script
15
+ * per hook.
16
16
  *
17
17
  * The second job is finding the runtime. The plugin payload is source and
18
18
  * manifests only — no `dist/`, no `node_modules/`, because a git-installed
@@ -88,7 +88,7 @@ function healInBackground() {
88
88
  const version = pinnedVersion();
89
89
  if (!version) return;
90
90
 
91
- // Claim the attempt BEFORE spawning. Four hooks fire per session and the
91
+ // Claim the attempt BEFORE spawning. Every hook fires per session and the
92
92
  // check alone would let every one of them start its own npm — a spawn storm
93
93
  // on the machine of the person whose install is already struggling. An hour
94
94
  // makes the claim self-expiring, so a heal that dies still gets retried
@@ -58,8 +58,9 @@ by what it is, not loaded by trigger.
58
58
  `eklavya` CLI and the read/write config tools; it does not teach or quiz.
59
59
  - **`agents/tutor.md`** is the subagent. It has the Eklavya MCP tools and
60
60
  read-only file access — and **no `AskUserQuestion`** — so it renders the four
61
- options as lettered text. Any change to the *Multiple choice* section of
62
- `skills/tutor/SKILL.md` has to hold for a plain-text renderer too.
61
+ options as lettered text. Any change to
62
+ `skills/tutor/references/writing-mcq.md` has to hold for a plain-text
63
+ renderer too.
63
64
 
64
65
  ## A skill is a prompt, but it is also an API client
65
66
 
@@ -148,23 +149,114 @@ cadence, so a tier there would sometimes name the previous question's
148
149
  difficulty, and a stale readout is worse than none. `level` covers what the tier
149
150
  was explaining: `easy` already means tiers 1-2.
150
151
 
151
- ## `skills/tutor/SKILL.md` has two readers
152
-
153
- `mcp/scripts/copy-assets.mjs` bundles it to `mcp/dist/assets/tutor-skill.md`,
154
- and `eklavya export-rules` (`mcp/src/cli.ts`) strips the frontmatter and wraps
155
- it as a Cursor rules file. So that file is consumed by two editors.
156
-
157
- Consequence: **no Claude-Code-only instructions in its body.** Slash-command
158
- names, plugin paths and hook mechanics belong in the command skills, not in
159
- the pedagogy. `AskUserQuestion` is the one unavoidable exception, and
160
- `agents/tutor.md` already carries the fallback for renderers that lack it.
152
+ ## The tutor skill is an entry point plus references
153
+
154
+ `skills/tutor/SKILL.md` was 5,296 words in one file, loaded whole whenever the
155
+ model decided a task was non-trivial. It is now the part that decides *whether
156
+ to act* — the log loop, checkpoint versus sweep, the shared budget, the tier
157
+ ladder, the plan's authoritative fields, and a Red Flags table of the
158
+ rationalizations that have each shipped a worse session — with the craft in
159
+ three siblings:
160
+
161
+ | File | Holds |
162
+ |---|---|
163
+ | `references/writing-mcq.md` | the four-option shape, `answer_position`, distractors, plain language, the second question about a concept, `prereqs_unmet`, and how to record a stem |
164
+ | `references/grading.md` | both scales, the mcq cap, feedback length, the four-step sequence a blank earns, `already_taught` |
165
+ | `references/focus-and-level.md` | the three focuses, the earned level bands, the cadence contract, the enforced-mode gate retry |
166
+
167
+ Two rules keep that split working.
168
+
169
+ **Mark a reference REQUIRED at the point of use, never as an `@`-link.** An
170
+ `@`-path is resolved eagerly by the host, which pulls the whole file into
171
+ context and undoes the split. "Read `references/grading.md` before you grade",
172
+ written where grading comes up, is what makes the model open it exactly when it
173
+ needs it.
174
+
175
+ **The entry point does not grow back, and the pointers stay honest.**
176
+ `test/packaging.test.ts` fails above 2,000 whitespace tokens, and asserts the
177
+ set of files named in SKILL.md is *equal* to the set on disk. Both directions
178
+ matter. A pointer with no file is the worse half — the model is told the rules
179
+ are elsewhere, cannot find them, and improvises, while nothing errors. A file
180
+ with no pointer is the quieter half: it ships, `export-rules` inlines it, and
181
+ Claude Code is never told to read it, so the same pedagogy differs by surface.
182
+ An earlier version of that test harvested pointers from SKILL.md *and*
183
+ `agents/tutor.md` into one list and asserted the list was non-empty — which
184
+ passed with no pointers in SKILL.md at all, the exact state it was written to
185
+ catch.
186
+
187
+ ## Match the form to the failure
188
+
189
+ Two failures need opposite wording, and using the wrong form measurably makes
190
+ things worse. superpowers A/B tested this on their own dispatch-prompt
191
+ guidance: the "don't do X" version produced **more** of the unwanted content
192
+ than the "here is the shape" version — the distributions fully separated — and
193
+ it did worse than giving no guidance at all.
194
+
195
+ - **The model knows the rule and breaks it under pressure.** Discipline. Ban
196
+ it, and name the excuse next to it: that is what the Red Flags table at the
197
+ top of `SKILL.md` is, and what the shared budget, one-question and
198
+ spent-question rules live in.
199
+ - **The model complies and produces the wrong shape.** Craft. Bans backfire
200
+ here. Describe the shape you want, in build order, and let the prohibitions
201
+ fall out of it as properties of the finished thing.
202
+
203
+ Writing a good multiple-choice question is the second kind, and
204
+ `references/writing-mcq.md` was written as the first kind — *never restate the
205
+ answer, no double negatives, avoid "which is NOT", do not number the options*.
206
+ It is now a six-part recipe in build order followed by a checklist of
207
+ properties, so the same rules arrive as "the answer appears among the options
208
+ and nowhere in the stem" rather than as separate bans to weigh.
209
+
210
+ **No nuance clauses in the recipe.** superpowers measured this separately: one
211
+ appended "unless it matters" turns a reliable recipe into a noisy one, because
212
+ it reopens the negotiation the recipe had settled. `writing-mcq.md` carried
213
+ exactly one — *"Save the precise term for when the precision is the point"* —
214
+ and it is gone. If an exception is real, it belongs in the plan's `framing`,
215
+ which is server-side and authoritative, not in a hedge the model gets to weigh.
216
+
217
+ `references/grading.md` keeps its prohibitions on purpose. Inflating a grade
218
+ and offering to stop because someone is blanking are pressure failures, not
219
+ shape failures: the model knows what honest grading is.
220
+
221
+ ## The tutor skill has two readers
222
+
223
+ `mcp/scripts/copy-assets.mjs` bundles the whole `skills/tutor/` directory to
224
+ `mcp/dist/assets/tutor/`, and `eklavya export-rules` (`mcp/src/cli.ts`) strips
225
+ the frontmatter and wraps it as a Cursor rules file. So the pedagogy is
226
+ consumed by two editors.
227
+
228
+ **Cursor has no progressive disclosure**, and that is the reason `export-rules`
229
+ concatenates SKILL.md with every `references/*.md` in alphabetical order and
230
+ says so in its preamble. A rules file is one document with `alwaysApply: true`,
231
+ so "read `references/grading.md`" there is a pointer to nothing. Had the split
232
+ shipped without the inlining, Cursor would have got the dispatch logic and none
233
+ of the craft — and every test would still have passed. `test/cli.test.ts` now
234
+ asserts a line from each reference reaches the output.
235
+
236
+ Alphabetical rather than a hand-kept order: in an always-apply document the
237
+ whole thing is in context at once, so order carries no meaning, and a listed
238
+ order is one more place a new reference gets forgotten.
239
+
240
+ **A missing reference is a hard failure there, not a warning.** `export-rules`
241
+ reads the pointers out of SKILL.md and refuses to emit anything if one of them
242
+ did not bundle, naming the file. It has to: the preamble promises the material
243
+ is further down the document, so a half-bundled export is worse than none — the
244
+ model is assured the rules are present and hunts for them instead of falling
245
+ back on what it has. `copy-assets.mjs` only warns when a copy fails, so that
246
+ state is reachable rather than hypothetical.
247
+
248
+ Consequence of the two readers: **no Claude-Code-only instructions in any of
249
+ the four files.** Slash-command names, plugin paths and hook mechanics belong
250
+ in the command skills, not in the pedagogy. `AskUserQuestion` is the one
251
+ unavoidable exception, and `agents/tutor.md` already carries the fallback for
252
+ renderers that lack it.
161
253
 
162
254
  ## Consistency
163
255
 
164
256
  The same behaviour described in two skills has drifted apart before — that is
165
257
  how `focus: project` got into three files. The four dials appear in
166
258
  `skills/mode/SKILL.md` and `user-skill/eklavya/SKILL.md`; the level bands
167
- appear in `skills/level/SKILL.md` and the *Level* section of `tutor`; the
168
- cadence cap appears in `mode`, `quiz` and `tutor`. **When you change one, grep
169
- the others for the same claim.** Where any of them disagrees with the code,
259
+ appear in `skills/level/SKILL.md` and `tutor/references/focus-and-level.md`;
260
+ the cadence cap appears in `mode`, `quiz` and that same reference. **When you
261
+ change one, grep the others for the same claim.** Where any of them disagrees with the code,
170
262
  `mcp/src/config.ts` and the tool file are right and the skill is wrong.
@@ -8,7 +8,7 @@ disable-model-invocation: true
8
8
 
9
9
  Get Eklavya working on this machine. Be brief; this should take one exchange.
10
10
 
11
- **1. Check prerequisites.** Run `node --version`. That is the whole list — the server, the CLI and all four hooks are Node, so nothing else has to be on `PATH`. Node must be 22+; below that the SQLite driver has no prebuilt binary and would need a C++ toolchain to install. If it is older, say so and how to upgrade on this platform, and stop: the rest of setup will not work.
11
+ **1. Check prerequisites.** Run `node --version`. That is the whole list — the server, the CLI and all five hooks are Node, so nothing else has to be on `PATH`. Node must be 22+; below that the SQLite driver has no prebuilt binary and would need a C++ toolchain to install. If it is older, say so and how to upgrade on this platform, and stop: the rest of setup will not work.
12
12
 
13
13
  Optionally run `eklavya doctor`, which reports the same thing plus the runtime, the SQLite driver, the plugin's registration, the chat skill, the database and the effective config. It exits non-zero and names the repair — `eklavya install` — if any of those has broken. That is also the command to reach for later, whenever Eklavya has gone quiet: the hooks never fail loudly, so a broken install looks exactly like a quiet one.
14
14