vimniscient 0.2.2__tar.gz → 0.3.0__tar.gz

{vimniscient-0.2.2 → vimniscient-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: vimniscient
-Version: 0.2.2
+Version: 0.3.0
 Summary: Get fluent in Vim in fifteen minutes a day, in your own Neovim.
 Project-URL: Homepage, https://vimniscient.com
 Project-URL: Source, https://github.com/dialectmaterial/Vimniscient
@@ -85,10 +85,8 @@ circles split by a line; `q` records a macro, a quill writing down what you
 do. Some come with a bit of history: `:t` copies lines because the old `ed`
 editor called a copy a *transfer*, and `grep` is named after `ed`'s `g/re/p`.
 
-The other texts are short facts about how memory and skill learning work
-(spacing, testing yourself, why mixed practice feels harder), the ideas
-Vimniscient itself is built on. Every one has a source in
-[`lessons/sources.md`](https://github.com/dialectmaterial/Vimniscient/blob/main/src/vimniscient/lessons/sources.md).
+Week 6 and a few code tasks are the exception: they're realistic on purpose,
+so that you practise on the kind of text you'll really edit.
 
 After you answer a card, and at the end of a lesson for the tasks you missed,
 Vimniscient shows what each part of the keys stands for:
@@ -187,9 +185,22 @@ lesson, and your progress with a keystroke-efficiency graph.
 | `Tab`, `j`/`k` | Move between panes and through lists |
 | `e` | Endless mode |
 | `c` / `g` / `p` | Cards / VimGolf / placement test |
+| `z` | Quick practice: 1, 2 or 5 minutes on one skill, navigation drills, or a multi-file mission |
+| `x` | Choose: edits with no hints, where you pick the strategy |
 | `s` | Stats |
 | `q` | Quit |
 
+## Is it working?
+
+`vimniscient eval pre` is a short test on texts from no lesson. Take it before
+you've trained much, then `vimniscient eval post` after some weeks and
+`vimniscient eval delayed` a few weeks after that. `vimniscient eval` compares
+them: solved, median time, and keys against the reference. Results stay on your
+machine.
+
+`vimniscient doctor` checks that Neovim and your config work with the trainer.
+`vimniscient placement reset` undoes a placement test.
+
 ## It adapts to your setup
 
 | You have | You get |
@@ -288,8 +299,8 @@ key stands for, so that the edit leaves it true: `ciw is change inner word:
 it swaps the wh▮ile word` (fix it with `ciw`). What each key officially means,
 and the hook to use where it means nothing, is in
 [`mnemonics.yaml`](https://github.com/dialectmaterial/Vimniscient/blob/main/src/vimniscient/mnemonics.yaml). The app reads the same
-file to say what a key stands for after you answer. Other sentences are about
-how memory and skill learning work; every such fact needs a source in
+file to say what a key stands for after you answer. Every task needs at least
+two texts. Memory science belongs in lesson intros, with a source in
 [`lessons/sources.md`](https://github.com/dialectmaterial/Vimniscient/blob/main/src/vimniscient/lessons/sources.md). Code tasks stay
 code, with the meaning in names and strings.
 
@@ -303,6 +314,10 @@ uv run python tests/check_timing.py             # task times: old progress datab
 uv run python tests/check_generate.py           # generated texts: reproducible, and they solve in Neovim
 uv run python tests/check_diagnose.py           # strategy feedback: no reference solution sets it off
 uv run python tests/check_transfer.py           # Choose: every way solves it in Neovim, nothing graded
+uv run python tests/check_quick.py              # quick practice: the right tasks for the time, nothing graded
+uv run python tests/check_missions.py           # missions: every way does the whole job on real files
+uv run python tests/check_placement.py          # placement, the evaluation and doctor
+uv run python tests/live_mission.py             # a mission in your real Neovim, in a hidden tmux session
 uv run python tests/check_today.py              # the daily budget, SRS scheduling, card introductions, hardtime
 uv run python tests/check_app.py                # every screen, driven headless
 uv run python tests/live_lessons.py --all       # types every solution into the real runner (needs tmux)

{vimniscient-0.2.2 → vimniscient-0.3.0}/README.md

@@ -62,10 +62,8 @@ circles split by a line; `q` records a macro, a quill writing down what you
 do. Some come with a bit of history: `:t` copies lines because the old `ed`
 editor called a copy a *transfer*, and `grep` is named after `ed`'s `g/re/p`.
 
-The other texts are short facts about how memory and skill learning work
-(spacing, testing yourself, why mixed practice feels harder), the ideas
-Vimniscient itself is built on. Every one has a source in
-[`lessons/sources.md`](https://github.com/dialectmaterial/Vimniscient/blob/main/src/vimniscient/lessons/sources.md).
+Week 6 and a few code tasks are the exception: they're realistic on purpose,
+so that you practise on the kind of text you'll really edit.
 
 After you answer a card, and at the end of a lesson for the tasks you missed,
 Vimniscient shows what each part of the keys stands for:
@@ -164,9 +162,22 @@ lesson, and your progress with a keystroke-efficiency graph.
 | `Tab`, `j`/`k` | Move between panes and through lists |
 | `e` | Endless mode |
 | `c` / `g` / `p` | Cards / VimGolf / placement test |
+| `z` | Quick practice: 1, 2 or 5 minutes on one skill, navigation drills, or a multi-file mission |
+| `x` | Choose: edits with no hints, where you pick the strategy |
 | `s` | Stats |
 | `q` | Quit |
 
+## Is it working?
+
+`vimniscient eval pre` is a short test on texts from no lesson. Take it before
+you've trained much, then `vimniscient eval post` after some weeks and
+`vimniscient eval delayed` a few weeks after that. `vimniscient eval` compares
+them: solved, median time, and keys against the reference. Results stay on your
+machine.
+
+`vimniscient doctor` checks that Neovim and your config work with the trainer.
+`vimniscient placement reset` undoes a placement test.
+
 ## It adapts to your setup
 
 | You have | You get |
@@ -265,8 +276,8 @@ key stands for, so that the edit leaves it true: `ciw is change inner word:
 it swaps the wh▮ile word` (fix it with `ciw`). What each key officially means,
 and the hook to use where it means nothing, is in
 [`mnemonics.yaml`](https://github.com/dialectmaterial/Vimniscient/blob/main/src/vimniscient/mnemonics.yaml). The app reads the same
-file to say what a key stands for after you answer. Other sentences are about
-how memory and skill learning work; every such fact needs a source in
+file to say what a key stands for after you answer. Every task needs at least
+two texts. Memory science belongs in lesson intros, with a source in
 [`lessons/sources.md`](https://github.com/dialectmaterial/Vimniscient/blob/main/src/vimniscient/lessons/sources.md). Code tasks stay
 code, with the meaning in names and strings.
 
@@ -280,6 +291,10 @@ uv run python tests/check_timing.py             # task times: old progress datab
 uv run python tests/check_generate.py           # generated texts: reproducible, and they solve in Neovim
 uv run python tests/check_diagnose.py           # strategy feedback: no reference solution sets it off
 uv run python tests/check_transfer.py           # Choose: every way solves it in Neovim, nothing graded
+uv run python tests/check_quick.py              # quick practice: the right tasks for the time, nothing graded
+uv run python tests/check_missions.py           # missions: every way does the whole job on real files
+uv run python tests/check_placement.py          # placement, the evaluation and doctor
+uv run python tests/live_mission.py             # a mission in your real Neovim, in a hidden tmux session
 uv run python tests/check_today.py              # the daily budget, SRS scheduling, card introductions, hardtime
 uv run python tests/check_app.py                # every screen, driven headless
 uv run python tests/live_lessons.py --all       # types every solution into the real runner (needs tmux)

{vimniscient-0.2.2 → vimniscient-0.3.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "vimniscient"
-version = "0.2.2"
+version = "0.3.0"
 description = "Get fluent in Vim in fifteen minutes a day, in your own Neovim."
 readme = "README.md"
 license = "AGPL-3.0-or-later"

{vimniscient-0.2.2 → vimniscient-0.3.0}/src/vimniscient/app.py

@@ -8,22 +8,24 @@ Omarchy theme in use (Catppuccin Mocha's own palette by default).
 
 from collections.abc import Callable
 from dataclasses import replace
+from datetime import datetime
 
 from textual.app import App
 from textual.color import Color
 from textual.content import Content
 
-from vimniscient import drills, endless, reminders, today, transfer, vimgolf
+from vimniscient import drills, endless, missions, placement, quick, reminders, today, transfer, vimgolf
 from vimniscient.cards import has_lazyvim, load_decks
 from vimniscient.lessons import Lesson, load_lessons, two_rounds
 from vimniscient.progress import Progress
-from vimniscient.session import COLORS, TaskResult, mouse_wheel, run_endless, run_lesson
-from vimniscient.ui.builtin_lessons import endless_lesson, placement_lesson, review_lesson, transfer_lesson, vimgolf_lesson
+from vimniscient.session import COLORS, TaskResult, mouse_wheel, run_endless, run_lesson, run_mission
+from vimniscient.ui.builtin_lessons import endless_lesson, mission_lesson, placement_lesson, quick_lesson, review_lesson, transfer_lesson, vimgolf_lesson
 from vimniscient.ui.cards import CardScreen
 from vimniscient.ui.common import CATPPUCCIN, DEFAULT_THEME, omarchy_theme, plural
 from vimniscient.ui.drills import DrillScreen
 from vimniscient.ui.lesson import LessonScreen, ResultsScreen
 from vimniscient.ui.main import MainScreen
+from vimniscient.ui.quick import QuickScreen
 from vimniscient.ui.stats import StatsScreen
 
 
@@ -45,7 +47,7 @@ class VimniscientApp(App):
 
     #main { height: 1fr; }
     #left { width: 46; max-width: 45%; }
-    #plan { height: 9; }
+    #plan { height: 10; }
     #lesson-list { height: 1fr; }
     #right { width: 1fr; }
     #detail-pane { height: 1fr; }
@@ -189,6 +191,8 @@ class VimniscientApp(App):
         elif step_id == "vimgolf":
             if not progress.vimgolf_done_today():
                 return self.start_vimgolf
+        elif step_id == "quick":
+            return self.start_quick
         elif step_id == "transfer":
             picks = transfer.pick(self.lessons, progress)
             if picks:
@@ -245,24 +249,79 @@ class VimniscientApp(App):
             self.notify("No cards due. Come back tomorrow.")
 
     def action_placement(self) -> None:
+        search = placement.Search(self.lessons)
+        self.push_screen(LessonScreen(placement_lesson(search.round()), start=lambda done: self.placement_round(search, done)))
+
+    def placement_round(self, search: placement.Search, done: Callable[[], None]) -> None:
+        tasks = search.round()
+
         def on_results(results: list[TaskResult]) -> str:
             self.progress.record(results)
-            placed = self.progress.place(self.lessons, results)
-            if not placed:
+            search.update(results)
+            search.seen |= {r.task.item_id for r in results}
+            if not search.finished:
+                return "That round is done. The next one follows from how it went."
+            known = search.known()
+            self.progress.mark_known(known, search.seen)
+            if not known:
                 return "No lessons skipped. Start from Week 1 · Day 1."
-            return "Known already: " + ", ".join(f"{p.label} ({p.title})" for p in placed)
+            return "Known already: " + ", ".join(f"{p.label} ({p.title})" for p in known)
 
-        self.push_screen(LessonScreen(placement_lesson(self.lessons), on_results))
+        def after() -> None:
+            self.placement_round(search, done) if not search.finished else done()
+
+        self.start_lesson(placement_lesson(tasks), on_results, after)
 
     def action_endless(self) -> None:
         self.run_step("endless", lambda: None)
 
+    def action_quick(self) -> None:
+        self.run_step("quick", lambda: None)
+
+    def start_quick(self, done: Callable[[], None]) -> None:
+        finished = {lesson.id for lesson in self.lessons if self.progress.is_done(lesson)}
+        pools = quick.pools(self.lessons, finished)
+        mission = missions.pick(self.lessons, self.progress)
+
+        def chosen(choice: tuple[str, int] | None) -> None:
+            if choice is None:
+                done()
+                return
+            skill, minutes = choice
+            if skill == "mission":
+                self.start_mission(mission, done)
+                return
+            count, seed = quick.how_many(minutes, self.progress.typical_task_seconds()), datetime.now().isoformat()
+            tasks = (quick.navigation(count, self.lessons[0].tasks[0], seed) if skill == quick.NAVIGATION
+                     else quick.tasks(pools[skill], count, seed))
+
+            def on_results(results: list[TaskResult]) -> str:
+                self.progress.record_ungraded(results, "quick")
+                return ""
+
+            self.start_lesson(quick_lesson(quick.title(skill), tasks), on_results, done)
+
+        self.push_screen(QuickScreen({skill: len(tasks) for skill, tasks in pools.items()}, mission.title if mission else ""), chosen)
+
+    def start_mission(self, mission: missions.Mission, done: Callable[[], None]) -> None:
+        with self.suspend():
+            result = run_mission(mission)
+        mouse_wheel(False)
+        self.progress.record_ungraded([result], "mission")
+        ways = "\n".join(mission.ways)
+        note = Content(f"{'Another way' if result.status == 'solved' else 'Ways to do it'}\n{ways}")
+
+        def after(again: bool) -> None:
+            self.start_mission(mission, done) if again else done()
+
+        self.push_screen(ResultsScreen(mission_lesson(mission), [result], note), after)
+
     def action_transfer(self) -> None:
         self.run_step("transfer", lambda: None)
 
     def start_transfer(self, picks: list[transfer.Challenge], done: Callable[[], None]) -> None:
         def on_results(results: list[TaskResult]) -> Content:
-            self.progress.record_transfer(results)
+            self.progress.record_ungraded(results, "transfer")
             by_goal = {c.task.goal: c for c in picks}
             lines = [f"{r.task.goal}  {'  or  '.join(ways)}" for r in results
                      if (ways := transfer.another_way(by_goal[r.task.goal], r.keys, r.status == "solved"))]

{vimniscient-0.2.2 → vimniscient-0.3.0}/src/vimniscient/cli.py

@@ -18,6 +18,11 @@ def main() -> None:
     remind.add_argument("--check", action="store_true", help="a follow-up reminder firing")
     budget = sub.add_parser("minutes", help="how long today's routine is meant to take (15 by default)")
     budget.add_argument("minutes", type=int, nargs="?", help="set it; leave it out to see it")
+    place = sub.add_parser("placement", help="undo the placement test")
+    place.add_argument("action", choices=["reset"])
+    sub.add_parser("doctor", help="check that Neovim and your setup work with the trainer")
+    evaluation = sub.add_parser("eval", help="measure your progress: a test now, one later, compared")
+    evaluation.add_argument("stage", choices=["pre", "post", "delayed", "report"], nargs="?", default="report")
     setup = sub.add_parser("reminders", help="daily practice reminders through Omarchy")
     setup.add_argument("action", choices=["install", "uninstall", "status"], nargs="?", default="status")
     args = parser.parse_args()
@@ -46,6 +51,14 @@ def main() -> None:
             settings.set_minutes(args.minutes)
         print(f"Today's routine takes about {settings.minutes()} minutes. ({settings.CONFIG})")
         return
+    if args.command == "doctor":
+        from vimniscient import doctor
+
+        ok = True
+        for check in doctor.checks():
+            print(f"{check.mark} {check.name}: {check.detail}")
+            ok = ok and (check.passed or not check.required)
+        raise SystemExit(0 if ok else 1)
     if args.command == "reminders":
         from vimniscient import reminders
 
@@ -69,6 +82,17 @@ def main() -> None:
     from vimniscient.progress import Progress
     from vimniscient.session import run_endless, run_lesson
 
+    if args.command == "placement":
+        print(f"Placement undone: {Progress().reset_placement()} lessons are back on the list.")
+        return
+    if args.command == "eval":
+        from vimniscient import evaluation as evaluate
+
+        progress = Progress()
+        if args.stage != "report":
+            progress.record_ungraded(run_lesson(evaluate.lesson(args.stage)), f"eval-{args.stage}")
+        print(evaluate.report(progress))
+        return
     if args.command == "endless":
         from vimniscient import endless
         from vimniscient.cards import has_lazyvim

vimniscient-0.3.0/src/vimniscient/doctor.py

@@ -0,0 +1,70 @@
+"""`vimniscient doctor`: does this machine work with the trainer? Required
+checks stop it working; the rest say what you'd be missing."""
+
+import os
+import re
+import shutil
+import sqlite3
+import subprocess
+from dataclasses import dataclass
+
+from vimniscient.cards import has_lazyvim, has_plugin
+
+MIN_NVIM = (0, 10)
+MIN_SIZE = (80, 24)
+
+
+@dataclass
+class Check:
+    name: str
+    passed: bool
+    detail: str
+    required: bool = True
+
+    @property
+    def mark(self) -> str:
+        return "✓" if self.passed else "✗" if self.required else "·"
+
+
+def _nvim() -> list[Check]:
+    if not shutil.which("nvim"):
+        return [Check("Neovim", False, f"not found; install {MIN_NVIM[0]}.{MIN_NVIM[1]} or newer")]
+    out = subprocess.run(["nvim", "--version"], capture_output=True, text=True).stdout
+    found = re.search(r"v(\d+)\.(\d+)", out)
+    version = (int(found[1]), int(found[2])) if found else (0, 0)
+    good = version >= MIN_NVIM
+    checks = [Check("Neovim", good, f"{version[0]}.{version[1]}" + ("" if good else f"; {MIN_NVIM[0]}.{MIN_NVIM[1]} or newer needed"))]
+    try:
+        run = subprocess.run(["nvim", "--headless", "-i", "NONE", "+qa"], capture_output=True, text=True, timeout=30)
+        problem = next((line for line in run.stderr.splitlines() if re.search(r"(?i)error", line)), "")
+        checks.append(Check("Your config", run.returncode == 0 and not problem, problem[:90] or "starts cleanly"))
+    except subprocess.TimeoutExpired:
+        checks.append(Check("Your config", False, "Neovim didn't start within 30 seconds"))
+    return checks
+
+
+def _terminal() -> Check:
+    size = shutil.get_terminal_size()
+    good = size.columns >= MIN_SIZE[0] and size.lines >= MIN_SIZE[1]
+    return Check("Terminal size", good, f"{size.columns}×{size.lines}" + ("" if good else f"; {MIN_SIZE[0]}×{MIN_SIZE[1]} or larger is needed"), required=False)
+
+
+def _data() -> Check:
+    from vimniscient.progress import DATA_DIR
+
+    path = DATA_DIR / "progress.db"
+    try:
+        DATA_DIR.mkdir(parents=True, exist_ok=True)
+        sqlite3.connect(path).execute("SELECT 1")
+        return Check("Progress data", True, str(path))
+    except (OSError, sqlite3.Error) as error:
+        return Check("Progress data", False, f"{path}: {error}")
+
+
+def checks() -> list[Check]:
+    optional = [
+        Check("LazyVim", has_lazyvim(), "week 5 lessons and the LazyVim cards" + ("" if has_lazyvim() else " stay hidden"), required=False),
+        Check("hardtime.nvim", has_plugin("hardtime.nvim"), "habit cards and drills", required=False),
+        Check("Hyprland", bool(os.environ.get("HYPRLAND_INSTANCE_SIGNATURE")), "Omarchy drills", required=False),
+    ]
+    return [*_nvim(), _data(), _terminal(), *optional]

vimniscient-0.3.0/src/vimniscient/eval.yaml

@@ -0,0 +1,136 @@
+# The evaluation: forms a and b are the same eight skills on different texts,
+# none of them from a lesson. `pre` uses a, `post` b, `delayed` a again, so a
+# change can't just be remembering the test. Realistic on purpose. Never graded.
+forms:
+  a:
+    - goal: 'Change the port to 9000.'
+      start: |-
+        ▮server = connect("db.local", 5432)
+      target: |-
+        server = connect("db.local", 9000)
+      ways: ['f5ciw9000<Esc>']
+    - goal: 'Delete the line with the password.'
+      start: |-
+        ▮user = "ana"
+        password = "hunter2"
+        host = "web1"
+      target: |-
+        user = "ana"
+        host = "web1"
+      ways: ['jdd']
+    - goal: 'Change "draft" to "final" in both places.'
+      start: |-
+        ▮status: draft
+        note: this is a draft
+      target: |-
+        status: final
+        note: this is a final
+      ways: [':%s/draft/final<CR>']
+    - goal: 'Put each name in quotes.'
+      start: |-
+        ▮anna
+        bert
+        carl
+      target: |-
+        "anna"
+        "bert"
+        "carl"
+      ways: [':%s/.*/"&"/<CR>']
+    - goal: 'Delete the arguments of the call.'
+      start: |-
+        ▮result = process(data, retries, timeout)
+      target: |-
+        result = process()
+      ways: ['f(di(']
+    - goal: 'Move the last line to the top.'
+      start: |-
+        one
+        two
+        ▮zero
+      target: |-
+        zero
+        one
+        two
+      ways: [':m0<CR>']
+    - goal: 'Sort the lines.'
+      start: |-
+        ▮pear
+        apple
+        mango
+      target: |-
+        apple
+        mango
+        pear
+      ways: [':sort<CR>']
+    - goal: 'Delete from here to the end of the line.'
+      start: |-
+        keep this▮, drop all of this
+      target: |-
+        keep this
+      ways: ['D']
+  b:
+    - goal: 'Change the timeout to 60.'
+      start: |-
+        ▮client = open("api.host", 30)
+      target: |-
+        client = open("api.host", 60)
+      ways: ['f3ciw60<Esc>']
+    - goal: 'Delete the line with the token.'
+      start: |-
+        ▮name = "bo"
+        token = "abc123"
+        path = "/srv"
+      target: |-
+        name = "bo"
+        path = "/srv"
+      ways: ['jdd']
+    - goal: 'Change "todo" to "done" in both places.'
+      start: |-
+        ▮state: todo
+        again: still todo
+      target: |-
+        state: done
+        again: still done
+      ways: [':%s/todo/done<CR>']
+    - goal: 'Put each word in quotes.'
+      start: |-
+        ▮red
+        green
+        blue
+      target: |-
+        "red"
+        "green"
+        "blue"
+      ways: [':%s/.*/"&"/<CR>']
+    - goal: 'Delete the arguments of the call.'
+      start: |-
+        ▮value = compute(left, right, scale)
+      target: |-
+        value = compute()
+      ways: ['f(di(']
+    - goal: 'Move the last line to the top.'
+      start: |-
+        beta
+        gamma
+        ▮alpha
+      target: |-
+        alpha
+        beta
+        gamma
+      ways: [':m0<CR>']
+    - goal: 'Sort the lines.'
+      start: |-
+        ▮walnut
+        almond
+        hazelnut
+      target: |-
+        almond
+        hazelnut
+        walnut
+      ways: [':sort<CR>']
+    - goal: 'Delete from here to the end of the line.'
+      start: |-
+        the good part▮, and the rest goes
+      target: |-
+        the good part
+      ways: ['D']

vimniscient-0.3.0/src/vimniscient/evaluation.py

@@ -0,0 +1,58 @@
+"""`vimniscient eval`: does the training work? Three sittings of the same eight
+skills, on texts from no lesson, with no hints: `pre` now, `post` after some
+weeks of training, `delayed` a few weeks after that. Stored locally, never graded.
+"""
+
+from functools import cache
+from importlib import resources
+from statistics import median
+
+import yaml
+
+from vimniscient.keys import count_keys
+from vimniscient.lessons import Lesson, Task, _parse_start
+
+LESSON_ID = "eval"
+HINT = "No hints in the test: :Skip what you don't know."
+FORM = {"pre": "a", "post": "b", "delayed": "a"}
+
+
+@cache
+def forms() -> dict[str, tuple[dict, ...]]:
+    data = yaml.safe_load((resources.files("vimniscient") / "eval.yaml").read_text(encoding="utf-8"))
+    return {name: tuple(tasks) for name, tasks in data["forms"].items()}
+
+
+def task(t: dict, index: int) -> Task:
+    start, cursor = _parse_start(t["start"])
+    return Task(goal=t["goal"], start=start, cursor=cursor, target=t["target"].split("\n"), solution="—", hint=HINT, ft="",
+                lesson_id=LESSON_ID, index=index, ref_override=min(count_keys(way) for way in t["ways"]))
+
+
+def lesson(stage: str) -> Lesson:
+    tasks = [task(t, i) for i, t in enumerate(forms()[FORM[stage]])]
+    return Lesson(id=LESSON_ID, week=0, day=0, title=f"Evaluation: {stage}", intro="", keys=[], tasks=tasks)
+
+
+def summary(rows: list[tuple[str, float | None, int, int]]) -> tuple[int, float | None, float | None]:
+    """(solved, median seconds, keys against the reference) from (status, seconds, count, ref) rows."""
+    solved = [row for row in rows if row[0] == "solved"]
+    seconds = [row[1] for row in solved if row[1] is not None]
+    keys = sum(row[2] for row in solved) / sum(row[3] for row in solved) if solved else None
+    return len(solved), median(seconds) if seconds else None, keys
+
+
+def report(progress) -> str:
+    lines = [f"{'':8}{'date':12}{'solved':8}{'median':9}keys/ref"]
+    for stage in FORM:
+        rows = progress.db.execute(
+            "SELECT status, seconds, count, ref, ts FROM attempts WHERE phase = ? ORDER BY id DESC LIMIT ?",
+            (f"eval-{stage}", len(forms()[FORM[stage]]))).fetchall()
+        if not rows:
+            lines.append(f"{stage:8}not taken yet: vimniscient eval {stage}")
+            continue
+        solved, seconds, keys = summary([row[:4] for row in rows])
+        time = f"{seconds:.1f}s" if seconds is not None else "-"
+        ratio = f"{keys:.2f}" if keys is not None else "-"
+        lines.append(f"{stage:8}{rows[0][4][:10]:12}{f'{solved}/{len(rows)}':8}{time:9}{ratio}")
+    return "\n".join(lines)