@polycode-projects/the-mechanical-code-talker 2.9.4 → 2.9.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -200,7 +200,7 @@ under a guard that proves the protected spans came through byte-for-byte.
|
|
|
200
200
|
|
|
201
201
|
A frozen regression suite plays out full multi-turn dialogues built from these
|
|
202
202
|
phrasings, from a single question up to a messy, typo-ridden real user.
|
|
203
|
-
Tier-by-tier detail is in `
|
|
203
|
+
Tier-by-tier detail is in `NEXT.md`.
|
|
204
204
|
|
|
205
205
|
## How it guides you
|
|
206
206
|
|
|
@@ -1082,7 +1082,7 @@ The full tables, judge scores, and transcripts are in the linked write-ups.
|
|
|
1082
1082
|
| Groundedness | Every answer carries a source, and an empty graph reports itself empty. Judge-scored mean 1.809/2 over 138 cases, 5 hard fails, 136/138 tier-1. | Judged (`claude-haiku-4-5-20251001`, `judge-prompt-v2`) at N=1. The judge prompt moved v1→v2 since 2.6.0, so this is a measurement, not a clean lever comparison against the prior cycle. The judge runs in the offline eval harness, never in the product. | `BENCHMARK_CEFR_ENGLISH_2.7.12.md` |
|
|
1083
1083
|
| Abstention (the honest miss) | 0% fabrication across 479 inference rows (379 chat + 100 kernel) and 0% hallucination across 272 agent rows | Structural, not a tuned threshold. tmct abstains because nothing matched, so the rows test a property of a no-model design rather than a score. | `BENCHMARK_INFERENCE_2.7.12.md`, `BENCHMARK_AGENT_2.7.12.md` |
|
|
1084
1084
|
| Determinism | Byte-identical on rerun — a 379-case `--replay` clean across 2 runs, no LLM, no network, $0 per turn | A property of the no-model pipeline. | `BENCHMARK_INFERENCE_2.7.12.md` |
|
|
1085
|
-
| Dialogue robustness (persona sweep) | A 6-persona sweep (textbook logician, casual newcomer, new developer, adversarial sceptic, returning user, planning user) fixed 25 of the prior cycle's 29 routed findings (21 clean, 4 with a residual noted); 4 remain broken, 2 in a shape distinct from the original complaint | Free exploration across all six personas surfaced roughly 60 fresh findings beyond the ratchet check — the single highest-signal pattern: tmct's own suggested repair text was itself frequently broken when followed verbatim (since fixed, see `
|
|
1085
|
+
| Dialogue robustness (persona sweep) | A 6-persona sweep (textbook logician, casual newcomer, new developer, adversarial sceptic, returning user, planning user) fixed 25 of the prior cycle's 29 routed findings (21 clean, 4 with a residual noted); 4 remain broken, 2 in a shape distinct from the original complaint | Free exploration across all six personas surfaced roughly 60 fresh findings beyond the ratchet check — the single highest-signal pattern: tmct's own suggested repair text was itself frequently broken when followed verbatim (since fixed, see `NEXT.md`). | `BENCHMARK_CONVERSATION_2.7.11.md` |
|
|
1086
1086
|
|
|
1087
1087
|
Three offline benchmark rigs live in a clone (they are not in the npm
|
|
1088
1088
|
package). Each replays a committed case set through the real product and
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@polycode-projects/the-mechanical-code-talker",
|
|
3
|
-
"version": "2.9.
|
|
3
|
+
"version": "2.9.6",
|
|
4
4
|
"private": false,
|
|
5
5
|
"type": "module",
|
|
6
6
|
"description": "The Mechanical Code Talker (tmct) — a tolerant, offline, $0 chat surface that guides you toward precision queries about a software repository. ELIZA/PARRY-style but domain-obsessed with code. No model calls; no codebase index of its own.",
|