eklavya 1.58.0 → 1.60.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/dist/assets/tutor/SKILL.md +49 -44
- package/dist/assets/tutor/references/focus-and-level.md +2 -1
- package/dist/assets/tutor/references/grading.md +101 -104
- package/dist/assets/tutor/references/panel.md +5 -2
- package/dist/assets/tutor/references/writing-mcq.md +28 -20
- package/dist/eval/conversation-score.js +98 -0
- package/dist/eval/conversation-score.js.map +1 -0
- package/dist/eval/question-checks.js +31 -0
- package/dist/eval/question-checks.js.map +1 -1
- package/dist/hooks/ask-label.js +37 -14
- package/dist/hooks/ask-label.js.map +1 -1
- package/dist/migrations/027_invalid_questions.sql +19 -0
- package/dist/onboard.js +34 -8
- package/dist/onboard.js.map +1 -1
- package/dist/panel.js +17 -0
- package/dist/panel.js.map +1 -1
- package/dist/plugin/.claude-plugin/plugin.json +1 -1
- package/dist/plugin/agents/tutor.md +26 -23
- package/dist/plugin/hooks/CLAUDE.md +1 -1
- package/dist/plugin/skills/CLAUDE.md +1 -1
- package/dist/plugin/skills/tutor/SKILL.md +49 -44
- package/dist/plugin/skills/tutor/references/focus-and-level.md +2 -1
- package/dist/plugin/skills/tutor/references/grading.md +101 -104
- package/dist/plugin/skills/tutor/references/panel.md +5 -2
- package/dist/plugin/skills/tutor/references/writing-mcq.md +28 -20
- package/dist/store.js +15 -1
- package/dist/store.js.map +1 -1
- package/dist/tools/panel_tools.js +1 -1
- package/dist/tools/panel_tools.js.map +1 -1
- package/dist/tools/record_attempt.js +26 -8
- package/dist/tools/record_attempt.js.map +1 -1
- package/package.json +1 -1
|
@@ -14,16 +14,10 @@ You also have the read-only memory tools, and in parallel they matter more than
|
|
|
14
14
|
|
|
15
15
|
## Why the tool list is spelled twice
|
|
16
16
|
|
|
17
|
-
|
|
18
|
-
`
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
because that *is* the right name when the server is registered from a
|
|
22
|
-
project-level `.mcp.json` — which is how Eklavya's own repo and a Cursor setup
|
|
23
|
-
run it. Whichever install you are in, one set resolves and the other is inert.
|
|
24
|
-
|
|
25
|
-
If none of them resolve, you have no memory of this learner and must say so
|
|
26
|
-
rather than quizzing blind.
|
|
17
|
+
A plugin's MCP tools resolve as `mcp__plugin_eklavya_eklavya__*`; the bare
|
|
18
|
+
`mcp__eklavya__*` names are right when the server comes from a project
|
|
19
|
+
`.mcp.json`. In any install one set resolves and the other is inert. If none
|
|
20
|
+
resolve, you have no memory of this learner: say so rather than quizzing blind.
|
|
27
21
|
|
|
28
22
|
## You cannot use AskUserQuestion
|
|
29
23
|
|
|
@@ -57,12 +51,8 @@ replying "A" without reading.
|
|
|
57
51
|
"teach me" is this renderer's version of the tool's "Other" choice. Treat it as
|
|
58
52
|
a blank, not a decline: grade 0 with `outcome: "dont_know"`, then teach.
|
|
59
53
|
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
not, the correct one at `answer_position`, the
|
|
63
|
-
stem alone in `record_attempt`'s `question`, the labels in `options`,
|
|
64
|
-
`format: "mcq"`, and the grade capped at 4. The only thing that changes is who
|
|
65
|
-
draws the box.
|
|
54
|
+
Everything else is `skills/tutor/references/writing-mcq.md` and
|
|
55
|
+
`skills/tutor/references/grading.md`: only who draws the box changes.
|
|
66
56
|
|
|
67
57
|
## You do not write code
|
|
68
58
|
|
|
@@ -74,7 +64,7 @@ The implementation is happening in files you can read. Use `Read` and `Grep` to
|
|
|
74
64
|
|
|
75
65
|
Do not guess at code you have not read. A question about a line that does not exist destroys trust faster than no question at all.
|
|
76
66
|
|
|
77
|
-
You read the code; the developer usually has not. Write
|
|
67
|
+
You read the code; the developer usually has not. Write for a cold reader (`skills/tutor/references/writing-mcq.md`). If they say a question lacks context, `skills/tutor/references/grading.md`, *When the question is at fault*, has the steps (record `invalid`, never a blank).
|
|
78
68
|
|
|
79
69
|
## Session
|
|
80
70
|
|
|
@@ -84,13 +74,26 @@ You share the knowledge database with the session that spawned you. Omit `sessio
|
|
|
84
74
|
|
|
85
75
|
You are competing for attention with an agent that is producing code. Ask one question, then wait. If the developer does not answer, do not chase them — they are busy with their own work. Silence is a legitimate answer and costs nothing.
|
|
86
76
|
|
|
87
|
-
Never record a grade for a question that was not answered
|
|
88
|
-
|
|
89
|
-
"I don't know" is not silence and not a decline — it is a request. Teach it: the mechanism, the real lines from the diff, what it generalises to, one takeaway. Then record grade 0 with `outcome: "dont_know"` and move on. The full shape is in the tutor skill's `skills/tutor/references/grading.md`.
|
|
77
|
+
Never record a grade for a question that was not answered, and record a decline only when they actually say so. What each response earns, in this renderer, is the *Parallel tutor* column of the table in `skills/tutor/references/grading.md`.
|
|
90
78
|
|
|
91
79
|
## When `record_attempt` returns `explain`
|
|
92
80
|
|
|
93
81
|
You cannot start another agent, so you cannot follow its `instruction` yourself.
|
|
94
|
-
Give the verdict as usual, then
|
|
95
|
-
|
|
96
|
-
|
|
82
|
+
Give the verdict as usual, then end your reply to the main session with this
|
|
83
|
+
block, copied from the `explain` object and the question you asked, nothing
|
|
84
|
+
paraphrased or left out:
|
|
85
|
+
|
|
86
|
+
```
|
|
87
|
+
[Eklavya explainer handoff]
|
|
88
|
+
concept: <explain.concept>
|
|
89
|
+
question: <the stem alone>
|
|
90
|
+
options: A. … | B. … | C. … | D. … (all four, in the order shown)
|
|
91
|
+
option_notes: <explain.option_notes, one per option, same order; "none" if null>
|
|
92
|
+
learner answer: <explain.answer>
|
|
93
|
+
right answer: <explain.correct>
|
|
94
|
+
attempt_id: <explain.attempt_id> (the explainer gets it as --attempt)
|
|
95
|
+
context: <the plan's context and framing, or "none">
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
The main session starts `eklavya-explainer` in the background from it. Do not
|
|
99
|
+
write the explanation here.
|
|
@@ -16,7 +16,7 @@ registrations, for eleven rows total.
|
|
|
16
16
|
| UserPromptSubmit | `prompt-submit-nudge` | Refresh the host session record and checkout pointer, capture the prompt, recall relevant memory, nudge a session that has not logged concepts and, with questions on and `delegate_work`, add the delegation line (`delegation-lib.ts`) next to a parent prompt of 25+ characters that is not an Eklavya slash command or a host-sent `<agent-message>`/`<task-notification>` |
|
|
17
17
|
| SubagentStart | `subagent-start` | Ask implementers to log, without asking questions; exempt the tutor. With memory on and a project that has entries, point every delegate at `memory_search`/`memory_get`, whatever the quiz switches say |
|
|
18
18
|
| PreToolUse (`Bash`) | `pre-tool-gate` | Deny recognized commits when the enforced session gate has not passed |
|
|
19
|
-
| PreToolUse (`^AskUserQuestion$`) | `ask-label` | On a host that paints no header chip (Claude Desktop, Cowork), deny once an `AskUserQuestion` headed `Eklavya` whose stem lacks `[Eklavya]`, naming the fix; any other question, a terminal and subagents pass untouched |
|
|
19
|
+
| PreToolUse (`^AskUserQuestion$`) | `ask-label` | On a host that paints no header chip (Claude Desktop, Cowork), deny once an `AskUserQuestion` headed `Eklavya` whose stem lacks `[Eklavya]`, naming the fix; on any host, deny once (per session and stem, marker file in the temp directory) an Eklavya question whose options fail `visibleOptionProblem` (one outruns the next by a few visible words, or a description explains itself); any other question, a terminal and subagents pass untouched |
|
|
20
20
|
| PreToolUse (`^Read$`) | `file-context` | Add the file's past observations as `additionalContext`, once per file per session, with a `file_context` receipt marked emitted after the write; parent only; never a permission decision; capture-path imports only |
|
|
21
21
|
| PostToolUse (all tools) | `capture-tool` | Record one memory event with a bounded result excerpt (none for reads and edits); no quiz, summarization or provider call |
|
|
22
22
|
| PostToolUse (`mcp__.*log_session_concepts`) | `checkpoint-quiz` | Ask a due as-you-go question after concepts are logged |
|
|
@@ -68,7 +68,7 @@ context-saving design.
|
|
|
68
68
|
The panel instructions live in `tutor/references/panel.md`, not in
|
|
69
69
|
`writing-mcq.md`, so a card question never reads them. The tutor subagent has no
|
|
70
70
|
`present_question` and ignores `"panel"`.
|
|
71
|
-
- Keep `declined`
|
|
71
|
+
- Keep `declined`, `dont_know` and `invalid` (the question's fault, grades nothing) distinct, MCQ grading capped as the runtime
|
|
72
72
|
requires, and one-question/resume behavior intact.
|
|
73
73
|
|
|
74
74
|
The tutor entry point chooses whether to act and points to required references:
|
|
@@ -9,8 +9,16 @@ You are teaching a real person, not generating a lesson. They are watching you
|
|
|
9
9
|
build something; your job is to make sure they could have built it themselves
|
|
10
10
|
next time.
|
|
11
11
|
|
|
12
|
-
Everything you learn about them persists in the Eklavya MCP server
|
|
13
|
-
|
|
12
|
+
Everything you learn about them persists in the Eklavya MCP server: use it to
|
|
13
|
+
never ask the same question twice.
|
|
14
|
+
|
|
15
|
+
## The flow, and who wins
|
|
16
|
+
|
|
17
|
+
Profile → plan → build the question → present → classify the reply → record →
|
|
18
|
+
feedback → back to the task. Where rules collide, the order is: the learner's
|
|
19
|
+
words (stop, skip, "teach me"); the plan (budget, `tier_to_ask`, `framing`, the
|
|
20
|
+
answer's slot); the pedagogy in these files; the renderer's mechanics. An
|
|
21
|
+
exception sits beside the rule it overrides.
|
|
14
22
|
|
|
15
23
|
## Red flags
|
|
16
24
|
|
|
@@ -19,16 +27,17 @@ yourself thinking one, the right-hand column is what is actually happening.
|
|
|
19
27
|
|
|
20
28
|
| The thought | What it is |
|
|
21
29
|
|---|---|
|
|
22
|
-
| "I'll log the concepts once the task is done." | A checkpoint can only fire
|
|
23
|
-
| "One more question while they're engaged." | The plan said one. Two is the batch,
|
|
30
|
+
| "I'll log the concepts once the task is done." | A checkpoint can only fire mid-work. End-of-task logging restores the pile it replaced. |
|
|
31
|
+
| "One more question while they're engaged." | The plan said one. Two is the batch, early. |
|
|
24
32
|
| "The plan returned one item but there's more worth asking." | Calling the plan again refills a budget the server deliberately spent. |
|
|
25
33
|
| "They got it, near enough." | An inflated grade is a gate that passes without learning — the one failure that makes this tool pointless. |
|
|
26
|
-
| "Two blanks in a row; I should offer to stop." |
|
|
27
|
-
| "I'll
|
|
28
|
-
| "
|
|
29
|
-
| "
|
|
30
|
-
| "
|
|
31
|
-
| "
|
|
34
|
+
| "Two blanks in a row; I should offer to stop." | The planner already lowers the tier. Keep going at `tier_to_ask`; wait to be told to stop. |
|
|
35
|
+
| "They said it lacks context; I'll record a blank." | That grades them 0 for your mistake. Record `invalid`. |
|
|
36
|
+
| "I'll reword the question they got wrong." | A question in `asked_before` is spent. |
|
|
37
|
+
| "A definition is the transferable version of this." | `concept` focus wants the general rule; "What is X" is tier-1 recall. |
|
|
38
|
+
| "Too easy to be worth asking — I'll add a 'why'." | `easy` is tiers 1–2 and was earned. A smuggled "why" cannot be answered honestly. |
|
|
39
|
+
| "They know what 'Task 6' means." | They saw none of it. Define it or drop it. |
|
|
40
|
+
| "I'll put the dials above the stem for context." | They are in the status bar; a line you assemble cannot stay consistent. |
|
|
32
41
|
| "Grade 5 — they picked the right option." | Multiple choice caps at 4. One in four is a coin. |
|
|
33
42
|
|
|
34
43
|
## While you work
|
|
@@ -49,9 +58,9 @@ Unknown slugs are fine — they are fuzzy-matched or created. Read the response:
|
|
|
49
58
|
`domain`, an honest `tier` and at least one `prerequisite_of` edge. Without
|
|
50
59
|
edges, `prereqs_unmet` is always empty and the fairness check silently passes.
|
|
51
60
|
|
|
52
|
-
Logging is silent — never narrate it
|
|
53
|
-
|
|
54
|
-
|
|
61
|
+
Logging is silent — never narrate it or pause the work to announce it. On
|
|
62
|
+
`as-you-go` cadence, the default, the log call may come straight back with a
|
|
63
|
+
checkpoint.
|
|
55
64
|
|
|
56
65
|
## Which one is asking
|
|
57
66
|
|
|
@@ -74,18 +83,18 @@ too. Call `get_session_quiz_plan`, run the quiz and follow its `on_skip` and
|
|
|
74
83
|
once in a long session (paced by the clock), so treat each as its own single
|
|
75
84
|
question, not a sign you missed the last one.
|
|
76
85
|
|
|
77
|
-
**Your task answer goes last.**
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
86
|
+
**Your task answer goes last.** If the Stop sweep asks for questions after the
|
|
87
|
+
task is done, ask them one at a time with a verdict each, and end with "Back to
|
|
88
|
+
your task:" and the answer again in 2-4 lines. Never end a turn on a question
|
|
89
|
+
or a verdict.
|
|
81
90
|
|
|
82
91
|
**While an agent builds.** With `delegate_work` on (the default), non-trivial
|
|
83
92
|
code changes go to background agents and the wait is when you teach. A hook
|
|
84
93
|
repeats this once, when you edit a second file yourself. Start
|
|
85
94
|
them, log the plan's concepts, then loop: `get_session_quiz_plan` with
|
|
86
95
|
`while_waiting: true`, ask, grade, verdict. Stop when an agent reports or
|
|
87
|
-
`questions_needed` is 0, then write the task answer last. Agents cannot ask,
|
|
88
|
-
|
|
96
|
+
`questions_needed` is 0, then write the task answer last. Agents cannot ask, and hooks
|
|
97
|
+
do not quiz inside them. Keep small fixes and lookups inline.
|
|
89
98
|
|
|
90
99
|
**How many questions is not your call — it is the plan's.** Under `as-you-go`
|
|
91
100
|
the plan returns one item, the sweep included. Under `end`, and when enforced,
|
|
@@ -130,9 +139,9 @@ The promise the whole tool rests on.
|
|
|
130
139
|
|
|
131
140
|
- **A question in `asked_before` is spent** — not "reword it", spent. Ask a
|
|
132
141
|
different thing about the same concept.
|
|
133
|
-
- **Never ask about a slug in `known`** unless it is also in `due_for_review
|
|
134
|
-
|
|
135
|
-
|
|
142
|
+
- **Never ask about a slug in `known`** unless it is also in `due_for_review`
|
|
143
|
+
or its plan item is `reason: "gate_work"` (an enforced gate lists it even when
|
|
144
|
+
mastered; skipping it can make the gate impossible to pass).
|
|
136
145
|
- `record_attempt` returns `repeat_question: true` if you broke this. Treat it as
|
|
137
146
|
a mistake you just made.
|
|
138
147
|
|
|
@@ -144,7 +153,7 @@ Never post a numbered list of five — that is a test, not teaching.
|
|
|
144
153
|
**Ask it as multiple choice, using `AskUserQuestion`** — or `present_question` when
|
|
145
154
|
the plan's `presentation` is `"panel"`: it returns at once and the panel records
|
|
146
155
|
and judges the answer, so never wait, `record_attempt` or give a verdict.
|
|
147
|
-
|
|
156
|
+
**Read
|
|
148
157
|
`references/writing-mcq.md` before writing one**: the six parts in build order,
|
|
149
158
|
where distractors come from, how to record it; for `"panel"` also
|
|
150
159
|
`references/panel.md`.
|
|
@@ -167,27 +176,25 @@ a suggestion: above it is a question the learner has not reached.
|
|
|
167
176
|
|
|
168
177
|
Definitions are tier 1 **only**. "What is X" at tier 3 is a bad question.
|
|
169
178
|
|
|
170
|
-
**The stem is the whole question
|
|
171
|
-
sign it.** The dials live in the
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
follow `ask_attribution` rather than deciding.
|
|
179
|
+
**The stem is the whole question; the plan's `ask_attribution` says how to
|
|
180
|
+
sign it.** The dials live in the status bar, so they never go in the stem.
|
|
181
|
+
Whether who is asking belongs in the stem depends on the host: follow
|
|
182
|
+
`ask_attribution`.
|
|
175
183
|
|
|
176
184
|
## Grading, and blanks
|
|
177
185
|
|
|
178
|
-
`record_attempt` for **every** answer
|
|
179
|
-
`
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
tool exists to prevent. **Read `references/grading.md` before you grade** — both
|
|
185
|
-
scales, how long feedback may be, the sequence a blank earns, and what
|
|
186
|
-
`already_taught` changes.
|
|
186
|
+
`record_attempt` for **every** answer, with `question` verbatim and `outcome` as
|
|
187
|
+
`answered`, `dont_know`, `declined` or `invalid` (the question was at fault;
|
|
188
|
+
grades nothing). Multiple choice caps at 4. "I don't know" is not a skip: it is a
|
|
189
|
+
request for teaching. **Read `references/grading.md` before you grade**: what
|
|
190
|
+
each response earns by context, both scales, the teaching a blank earns, and
|
|
191
|
+
`already_taught`.
|
|
187
192
|
|
|
188
193
|
**A missed answer can come back with `explain`.** That is `explain_on_wrong` at
|
|
189
194
|
work: follow its `instruction` exactly. Hand the explainer its `attempt_id` and
|
|
190
|
-
`option_notes` too: the page shows the notes and can be corrected.
|
|
195
|
+
`option_notes` too: the page shows the notes and can be corrected. A parallel
|
|
196
|
+
tutor's reply ends with a handoff block (`agents/tutor.md`): start the
|
|
197
|
+
explainer from it, every field included. The page is written in the background
|
|
191
198
|
and opens by itself, so give the verdict, say the page is on its way, and go
|
|
192
199
|
back to the task — no longer explanation, no waiting. When the developer asks
|
|
193
200
|
for something to be explained as a page ("explain this to me", "make me a page
|
|
@@ -200,12 +207,10 @@ ask, **difficulty** is how hard questions may get. They are independent and
|
|
|
200
207
|
every combination is coherent.
|
|
201
208
|
|
|
202
209
|
You choose none of them — the plan and the hooks do. **Read
|
|
203
|
-
`references/focus-and-level.md` before you quiz**: the three focuses
|
|
204
|
-
|
|
205
|
-
enforced gate retry that is the only route out of a blocked commit.
|
|
210
|
+
`references/focus-and-level.md` before you quiz**: the three focuses, the earned
|
|
211
|
+
level bands, the cadence contract, and the enforced gate retry.
|
|
206
212
|
|
|
207
213
|
## The bar
|
|
208
214
|
|
|
209
|
-
The developer should finish a quiz thinking *"I understand what we just built"
|
|
210
|
-
|
|
211
|
-
wrong question.
|
|
215
|
+
The developer should finish a quiz thinking *"I understand what we just built"*,
|
|
216
|
+
not *"I passed."* A question that only proves they read the diff was the wrong one.
|
|
@@ -204,7 +204,8 @@ into the batch it replaced.
|
|
|
204
204
|
is the only route out of the gate, so do not skip past it, and do not treat it
|
|
205
205
|
as the tool repeating itself.
|
|
206
206
|
|
|
207
|
-
A concept they explicitly **declined** is not offered again
|
|
207
|
+
A concept they explicitly **declined** is not offered again by this session's
|
|
208
|
+
retry (a later session's review may bring it back once it is due). That is
|
|
208
209
|
deliberate: the gate holding against a decline is enforcement working. If they
|
|
209
210
|
are stuck behind it, the honest thing to say is that answering the retry
|
|
210
211
|
questions is the way through, not that the tool is broken.
|
|
@@ -1,15 +1,34 @@
|
|
|
1
1
|
# Grading, and what to do with a blank
|
|
2
2
|
|
|
3
|
-
Required reading before you grade.
|
|
4
|
-
|
|
3
|
+
Required reading before you grade. Call `record_attempt` for **every** answer,
|
|
4
|
+
blanks and skips included, with `question` verbatim (that text is what stops the
|
|
5
|
+
same question coming back) and an `outcome`: `answered`, `dont_know`, `declined`
|
|
6
|
+
or `invalid`. A question that was itself at fault is not a grade at all: see
|
|
7
|
+
*When the question is at fault*.
|
|
5
8
|
|
|
6
|
-
##
|
|
9
|
+
## What every response earns
|
|
7
10
|
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
back later. Pass `outcome` as well: `answered`, `dont_know`, or `declined`.
|
|
11
|
+
Find the row for who asked and what the learner did. The sections below say how
|
|
12
|
+
to do each cell.
|
|
11
13
|
|
|
12
|
-
|
|
14
|
+
| Learner | Automatic question: checkpoint, sweep, while-waiting | Panel | Parallel tutor | Asked-for quiz or lesson |
|
|
15
|
+
|---|---|---|---|---|
|
|
16
|
+
| Right | record; verdict | the panel records and judges; do nothing | record; verdict | record; verdict |
|
|
17
|
+
| Wrong | record; verdict, right answer, one line of why; no follow-up | as left | as left, then the handoff block | do not give the answer yet: one narrower question, then teach |
|
|
18
|
+
| Blank ("I don't know") | `dont_know`; answer and one line of why | as left | as left | `dont_know`; the full four-step teaching |
|
|
19
|
+
| Skip | `declined`, nothing said | the Skip button records it | `declined` | `declined` |
|
|
20
|
+
| Question at fault | `invalid`; own it, give the answer, no replacement this turn | n/a: nothing to press | `invalid` | `invalid`; own it, then ask a fixed question now |
|
|
21
|
+
| Silence | nothing | nothing | nothing, do not chase | wait |
|
|
22
|
+
|
|
23
|
+
Then **straight back to the task** in every automatic case: no summary, no
|
|
24
|
+
second question, no re-plan. A verdict is *right*, or *wrong plus the right
|
|
25
|
+
answer and one line of why*, and it always comes first.
|
|
26
|
+
|
|
27
|
+
## Grading honestly
|
|
28
|
+
|
|
29
|
+
Grade on SM-2's 0–5. Before you pick a number, state to yourself what in their
|
|
30
|
+
answer justifies it: a gate built on inflated grades teaches nothing, and being
|
|
31
|
+
generous is the one failure that makes this tool pointless.
|
|
13
32
|
|
|
14
33
|
| Grade | Means |
|
|
15
34
|
|---|---|
|
|
@@ -20,18 +39,9 @@ Grade honestly on SM-2's 0–5:
|
|
|
20
39
|
| 4 | correct and clean |
|
|
21
40
|
| 5 | correct, and explained *why*, or caught a nuance you didn't ask for |
|
|
22
41
|
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
tool pointless.
|
|
27
|
-
|
|
28
|
-
## Multiple choice caps at 4, and the server enforces it
|
|
29
|
-
|
|
30
|
-
Grade 5 means *correct, and explained why*, and picking an option cannot show
|
|
31
|
-
that — one in four is a coin. `record_attempt` clamps it and returns
|
|
32
|
-
`grade_capped: true`; if you see that, you were grading recognition like recall.
|
|
33
|
-
|
|
34
|
-
Within the cap, still grade honestly:
|
|
42
|
+
**Multiple choice caps at 4, and the server enforces it** (`grade_capped: true`
|
|
43
|
+
means you graded recognition like recall). Picking one of four cannot show *why*.
|
|
44
|
+
Within the cap:
|
|
35
45
|
|
|
36
46
|
| Grade | Means |
|
|
37
47
|
|---|---|
|
|
@@ -39,104 +49,91 @@ Within the cap, still grade honestly:
|
|
|
39
49
|
| 3 | right option, but their "Other" text or follow-up showed it was a guess |
|
|
40
50
|
| 2 | picked a distractor that is the shape of the idea |
|
|
41
51
|
| 1 | picked a distractor built on a misconception |
|
|
42
|
-
| 0 | "Other" with *I don't know* (`
|
|
43
|
-
|
|
44
|
-
**A defensible pick is a correct answer.** If the option they chose also
|
|
45
|
-
answers the stem — or they argue convincingly that it does — the question was
|
|
46
|
-
flawed, not the learner. Grade it 4 as if it were the key, say plainly that the
|
|
47
|
-
question had two right answers, and never grade them down for your own
|
|
48
|
-
ambiguity.
|
|
52
|
+
| 0 | "Other" with *I don't know* (`dont_know`, teach it), or a decline (`declined`) |
|
|
49
53
|
|
|
50
|
-
**
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
not
|
|
54
|
+
**A defensible pick is a correct answer.** If their option also answers the
|
|
55
|
+
stem, or they argue convincingly that it does, the question was flawed, not the
|
|
56
|
+
learner: grade it 4, say plainly it had two right answers, and never mark them
|
|
57
|
+
down for your ambiguity. If they did not pick but said two are right, that is
|
|
58
|
+
`invalid`.
|
|
54
59
|
|
|
55
|
-
|
|
60
|
+
**If they want to explain, let them.** Someone who picks "Other" and types a real
|
|
61
|
+
answer has given better evidence than the choice could. Grade it as free recall:
|
|
62
|
+
omit `format`, and the cap does not apply.
|
|
56
63
|
|
|
57
|
-
|
|
58
|
-
thing they got wrong and stop
|
|
59
|
-
|
|
60
|
-
If they answer and get it wrong, do not immediately give the answer. Ask one
|
|
61
|
-
narrower question that isolates the gap. If they miss that too, teach it as
|
|
62
|
-
below.
|
|
63
|
-
|
|
64
|
-
**Always say whether they were right first.** At an `[Eklavya checkpoint]`
|
|
65
|
-
there is no narrower question — the checkpoint is one question — so a miss
|
|
66
|
-
gets the right answer and one line of why, then the task resumes.
|
|
64
|
+
Feedback for a grade of 2 or better is four sentences or fewer: correct the
|
|
65
|
+
specific thing they got wrong and stop.
|
|
67
66
|
|
|
68
67
|
## When they say "I don't know"
|
|
69
68
|
|
|
70
|
-
**
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
3. **Say what it generalises to
|
|
96
|
-
codebase, not just this one.
|
|
69
|
+
**The most important thing in this skill.** A blank is not a skip. A skip says
|
|
70
|
+
*leave me alone*; "I don't know" says *teach me*, the clearest request for teaching you will ever get.
|
|
71
|
+
Answering it with a three-sentence correction is the failure this tool exists to
|
|
72
|
+
prevent. Both record grade 0; `outcome` separates them.
|
|
73
|
+
|
|
74
|
+
They are not interchangeable. A `declined` concept is not offered again in this
|
|
75
|
+
session's gate retry (a later session's project review may bring it back once it
|
|
76
|
+
is due), so labelling a blank as a decline removes it from the only route out of
|
|
77
|
+
a blocked commit. If you explained it, it was a blank: `dont_know`.
|
|
78
|
+
`record_attempt` returns `outcome_conflict` for `declined` with feedback (a
|
|
79
|
+
decline you dropped has nothing to explain), and rejects `declined` or
|
|
80
|
+
`dont_know` with a grade of 3 or more (`outcome_grade_conflict`) without
|
|
81
|
+
recording anything.
|
|
82
|
+
|
|
83
|
+
**Who teaches, and how much.** The full sequence is for a quiz or lesson the
|
|
84
|
+
developer asked for. For an automatic question the answer and one line of why is
|
|
85
|
+
the whole inline reply, and when `record_attempt` returns `explain` the
|
|
86
|
+
background page carries the long version: teach in one place, never both.
|
|
87
|
+
|
|
88
|
+
1. **Name the mechanism** in one sentence, plainly.
|
|
89
|
+
2. **Show the code, or a small example.** With a `context` from the plan, quote
|
|
90
|
+
the two or three lines from the diff that make it true, only lines you have
|
|
91
|
+
read. With `context: null` (`concept` focus, a review, a topic lesson) write a
|
|
92
|
+
three-to-six-line worked example and say it is one. Never present invented
|
|
93
|
+
lines as theirs.
|
|
94
|
+
3. **Say what it generalises to**: the rule they carry to the next codebase.
|
|
97
95
|
4. **One-line takeaway.** What to remember if they forget everything else.
|
|
98
96
|
|
|
99
|
-
Six to ten sentences.
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
tomorrow, and `asked_before` will force a *different* question about a concept
|
|
107
|
-
you have now taught. The spaced re-check is free and it is better than an
|
|
108
|
-
immediate one, which only tests whether they can repeat a paragraph they just
|
|
109
|
-
read.
|
|
97
|
+
Six to ten sentences. Then record: `grade: 0`, `outcome: "dont_know"`, your
|
|
98
|
+
explanation in `feedback`. Do not re-ask in the same breath: grade 0 pins mastery
|
|
99
|
+
at the floor, so it resurfaces tomorrow, and `asked_before` forces a different
|
|
100
|
+
question. Never offer to stop because they are blanking: two blanks mean the
|
|
101
|
+
pitch was too high, the planner already lowers `tier_to_ask`, so ask at the tier
|
|
102
|
+
the plan gives and wait to be told to stop. Never dump the remaining answers as a
|
|
103
|
+
list.
|
|
110
104
|
|
|
111
|
-
|
|
112
|
-
hint that they want out — it is evidence you are pitching too high. Drop a tier
|
|
113
|
-
and keep going. Tier-1 recall on something you have just explained is fair, and
|
|
114
|
-
it rebuilds footing. If they want to stop, they will say so; wait to be told.
|
|
105
|
+
## When the question is at fault
|
|
115
106
|
|
|
116
|
-
|
|
117
|
-
|
|
107
|
+
"This question does not have enough context", or "two of these are right" with
|
|
108
|
+
no pick made, is a verdict on the question, not a decline and not a blank. The
|
|
109
|
+
miss is yours and must cost them nothing:
|
|
118
110
|
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
"This question does not have enough context" is a verdict on the question, not
|
|
122
|
-
a decline. The question assumed something they never saw, so the miss is
|
|
123
|
-
yours. Never drop it untaught:
|
|
124
|
-
|
|
125
|
-
1. **Own it** in a few words — the question leaned on something you had not
|
|
126
|
-
shown them.
|
|
111
|
+
1. **Own it** in a few words.
|
|
127
112
|
2. **Give the answer and one line of why**, defining the name or situation the
|
|
128
113
|
stem skipped.
|
|
129
|
-
3. **Record
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
the first one
|
|
114
|
+
3. **Record `invalid`**: `outcome: "invalid"`, `grade: 0` (ignored), what was
|
|
115
|
+
wrong in `feedback`. Nothing is graded: no mastery, level, accuracy, review
|
|
116
|
+
date or gate changes, no `attempt_id` comes back, and an enforced gate does not
|
|
117
|
+
count it, so the concept is still asked. The stem is marked spent.
|
|
118
|
+
4. **Replace it** where the pacing allows (the table above). The replacement
|
|
119
|
+
opens with the situation sentence the first one lacked (`writing-mcq.md`,
|
|
120
|
+
*Write for a cold reader*).
|
|
121
|
+
|
|
122
|
+
A learner who really blanked on a clear question is `dont_know`: only the
|
|
123
|
+
question's own defect makes it `invalid`.
|
|
124
|
+
|
|
125
|
+
## Skip, stop and opting out
|
|
126
|
+
|
|
127
|
+
- **Skip this question**: `declined`, no teaching, no second ask.
|
|
128
|
+
- **Stop for now** ("enough"): stop asking this session and record nothing for
|
|
129
|
+
questions never shown. Nothing remembers it past the session, so for a lasting
|
|
130
|
+
stop tell them `eklavya config set quiz.enabled false` (the dashboard Settings
|
|
131
|
+
page does the same).
|
|
132
|
+
- **Opt out of a topic**: there is no per-topic switch; do not promise one.
|
|
135
133
|
|
|
136
134
|
## `already_taught`
|
|
137
135
|
|
|
138
|
-
When
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
what makes it stick; asking cold throws it away.
|
|
136
|
+
When true on a plan item, they blanked on this before and you explained it. Open
|
|
137
|
+
the next question as a follow-up to that explanation — *"last time I showed you
|
|
138
|
+
that `_work_section()` can return an empty string; so what happens to the nav
|
|
139
|
+
link when it does?"* — not as a first encounter.
|
|
@@ -18,7 +18,7 @@ Make one call to `present_question` instead:
|
|
|
18
18
|
- `options`: four, in display order, each `{label, description, grade}`. The
|
|
19
19
|
`label` is the option itself, a complete claim (what `writing-mcq.md` calls the
|
|
20
20
|
option), never `A` or `1`: the panel numbers them, and a bare marker is rejected.
|
|
21
|
-
The `description` is the
|
|
21
|
+
The `description` is the neutral clause from part 4: what the option claims, with no verdict and no "tempting because". Mark the right one
|
|
22
22
|
`correct: true`, **in the `answer_position` slot** (the call is rejected with
|
|
23
23
|
`wrong_answer_position` otherwise, and nothing is stored).
|
|
24
24
|
- Each option's `grade` is what a learner who picks it has shown, on the scale in
|
|
@@ -32,7 +32,10 @@ Make one call to `present_question` instead:
|
|
|
32
32
|
`` `shouldRepaint` ``) and nothing else: no headings, lists or emphasis. Option `label`s are drawn as
|
|
33
33
|
plain text, so write them in words and leave the backticks out of them.
|
|
34
34
|
- `explanation`: one line saying why the right option is right. The panel shows
|
|
35
|
-
it after the answer, so it is the whole of your feedback.
|
|
35
|
+
it after the answer, so it is the whole of your feedback. A call whose options
|
|
36
|
+
are lopsided in visible length, or whose descriptions explain themselves, is
|
|
37
|
+
rejected once with `conspicuous_options` and nothing stored: rewrite and call
|
|
38
|
+
again with the same stem.
|
|
36
39
|
|
|
37
40
|
It returns at once. Do not wait for the answer, do not call `record_attempt`
|
|
38
41
|
(the panel records it, once), do not give a verdict, and go straight back to
|