misakanet 2.50.0 → 3.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.codex-plugin/plugin.json +1 -1
- package/README.ja.md +25 -5
- package/README.md +64 -12
- package/README.zh-CN.md +23 -5
- package/lib/client.js +62 -3
- package/package.json +2 -2
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "misakanet",
|
|
3
|
-
"version": "
|
|
3
|
+
"version": "3.0.0",
|
|
4
4
|
"description": "Failure-memory network for AI coding agents: search evidence-rated failure lessons, submit new failures, and turn crashes into reusable knowledge.",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "Ikalus1988",
|
package/README.ja.md
CHANGED
|
@@ -39,7 +39,7 @@ mcp-name: io.github.Ikalus1988/misakanet
|
|
|
39
39
|
<p align="center">
|
|
40
40
|
<em>Install</em>
|
|
41
41
|
|
|
42
|
-
<a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/python-3.
|
|
42
|
+
<a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/python-3.11+-blue" alt="Python"></a>
|
|
43
43
|
<a href="https://pypi.org/project/misakanet/"><img src="https://img.shields.io/pypi/v/misakanet" alt="PyPI"></a>
|
|
44
44
|
<a href="https://www.npmjs.com/package/misakanet"><img src="https://img.shields.io/npm/v/misakanet" alt="npm"></a>
|
|
45
45
|
<a href="https://github.com/marketplace/actions/misakanet-intake-bot"><img src="https://img.shields.io/badge/Marketplace-MisakaNet%20Intake%20Bot-blue?logo=github" alt="GitHub Marketplace"></a>
|
|
@@ -50,6 +50,7 @@ mcp-name: io.github.Ikalus1988/misakanet
|
|
|
50
50
|
|
|
51
51
|
<a href="https://glama.ai/mcp/connectors/org.misakanet/misaka-net"><img src="https://glama.ai/mcp/connectors/org.misakanet/misaka-net/badges/score.svg" alt="MisakaNet MCP connector on Glama"></a>
|
|
52
52
|
<a href="https://mcptoplist.com/server/io.github.Ikalus1988%2Fmisakanet"><img src="https://img.shields.io/badge/MCP-Toplist-blue" alt="MCP Toplist"></a>
|
|
53
|
+
<a href="https://dshfind.com/en/plugins/Ikalus1988/MisakaNet?ref=badge"><img src="https://dshfind.com/api/badge/Ikalus1988/MisakaNet?metric=downloads&lang=en" alt="dshfind の DSH プラグイン ダウンロード数"></a>
|
|
53
54
|
</p>
|
|
54
55
|
|
|
55
56
|
---
|
|
@@ -107,9 +108,9 @@ intake を返します——「この失敗はまだ知らない」というコ
|
|
|
107
108
|
|---|---|
|
|
108
109
|
| ❌ 汎用メモリ | ✅ 失敗回復の知識レイヤー |
|
|
109
110
|
| ❌ エージェントランタイム | ✅ 検索できるレッスンデータベース |
|
|
110
|
-
| ❌ ベクトル DB / RAG スタック | ✅ BM25 キーワード検索、標準ライブラリのみ(Python ≥ 3.
|
|
111
|
+
| ❌ ベクトル DB / RAG スタック | ✅ BM25 キーワード検索、標準ライブラリのみ(Python ≥ 3.11 は必要) |
|
|
111
112
|
| ❌ スキルマーケット | ✅ 実際のセッションから出たデバッグ知識 |
|
|
112
|
-
| ❌
|
|
113
|
+
| ❌ 汎用ベンチマーク基盤 | ✅ 証拠で格付けされたコーパス。267 クエリの検索評価を公開、旧復唱ベンチマークは停止中 |
|
|
113
114
|
|
|
114
115
|
### 何に答え、何に答えないか
|
|
115
116
|
|
|
@@ -135,7 +136,26 @@ MisakaNet は索引済みの失敗にだけ答えます。一致が無いとき
|
|
|
135
136
|
intake を返します。
|
|
136
137
|
- **報告の速度がトリアージの速度を上回っています**:intake キューが正直なボトルネックです。
|
|
137
138
|
|
|
138
|
-
##
|
|
139
|
+
## 検索ベンチマーク:267 クエリ、BM25 vs semantic fallback
|
|
140
|
+
|
|
141
|
+
2026-10-27、hosted endpoint の検索経路を **267 クエリ**(レッスン 1 件につき 1 クエリ、Workers AI
|
|
142
|
+
70B が生成)で評価しました。両 arm は同じ eval worker / corpus を使い、`MISAKANET_SEMANTIC` だけを
|
|
143
|
+
切り替えています。
|
|
144
|
+
|
|
145
|
+
| arm | R@1 | R@5 | no_match |
|
|
146
|
+
|---|---|---|---|
|
|
147
|
+
| BM25 | 0.210 | 0.225 | 0.712 |
|
|
148
|
+
| semantic fallback | **0.446** | **0.472** | **0.453** |
|
|
149
|
+
|
|
150
|
+
semantic fallback は R@1 を **23.6 ポイント**上げ、`no_match` を **25.9 ポイント**下げました。ただし
|
|
151
|
+
R@1 は **0.446 < 0.46** で R@1 条件は未達、`no_match` は **0.453 ≤ 0.50** で no_match 条件は達成です。
|
|
152
|
+
113 クエリの subset は 0.460/0.460、250 クエリは 0.404/0.504——fallback が効くことは示せますが、
|
|
153
|
+
R@1 が解決したと宣言できるサンプルではありません。
|
|
154
|
+
|
|
155
|
+
これは 300 レッスン中の 267 クエリで、650 全量ではありません。33 レッスンはクエリを生成できず、
|
|
156
|
+
semantic 側は 2 件エラー、eval corpus も未更新です。→ [全文レポート](docs/benchmarks/retrieval-l1-267-2026-10-27.md)
|
|
157
|
+
|
|
158
|
+
## 復唱ベンチマーク:何を測り、なぜ停止したか
|
|
139
159
|
|
|
140
160
|
測ったのは 1 つの狭い問いだけです:**レッスンをモデルに渡したとき、そのレッスンのどれだけが返ってくるか。**
|
|
141
161
|
指標は `lesson_hit_rate`——注入したレッスンのコマンドが回答に再現された割合です。数字の前に設定を読んで
|
|
@@ -222,7 +242,7 @@ with_lesson hit rate (per run date)
|
|
|
222
242
|
([理由](AGENTS.md#33-注册与配额))。
|
|
223
243
|
|
|
224
244
|
インストールの確認:`npx @misaka-net/misakanet-setup --verify`、取り消し:`--uninstall`。
|
|
225
|
-
前提:インストーラーは Node ≥ 18、ライブラリと stdio サーバーは Python ≥ 3.10
|
|
245
|
+
前提:インストーラーは Node ≥ 18、ライブラリと stdio サーバーは Python ≥ 3.11。Python 3.10 は 2026-10-01 でサポート終了(EOL)になったため、このリリースには 3.11 以降が必要です——パッケージを更新する前にインタプリタを更新してください。
|
|
226
246
|
|
|
227
247
|
### GitHub Action として使う
|
|
228
248
|
|
package/README.md
CHANGED
|
@@ -39,7 +39,7 @@ A2A discovery via `.well-known/agent-card.json`.
|
|
|
39
39
|
<p align="center">
|
|
40
40
|
<em>Install</em>
|
|
41
41
|
|
|
42
|
-
<a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/python-3.
|
|
42
|
+
<a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/python-3.11+-blue" alt="Python"></a>
|
|
43
43
|
<a href="https://pypi.org/project/misakanet/"><img src="https://img.shields.io/pypi/v/misakanet" alt="PyPI"></a>
|
|
44
44
|
<a href="https://www.npmjs.com/package/misakanet"><img src="https://img.shields.io/npm/v/misakanet" alt="npm"></a>
|
|
45
45
|
<a href="https://github.com/marketplace/actions/misakanet-intake-bot"><img src="https://img.shields.io/badge/Marketplace-MisakaNet%20Intake%20Bot-blue?logo=github" alt="GitHub Marketplace"></a>
|
|
@@ -50,6 +50,7 @@ A2A discovery via `.well-known/agent-card.json`.
|
|
|
50
50
|
|
|
51
51
|
<a href="https://glama.ai/mcp/connectors/org.misakanet/misaka-net"><img src="https://glama.ai/mcp/connectors/org.misakanet/misaka-net/badges/score.svg" alt="MisakaNet MCP connector on Glama"></a>
|
|
52
52
|
<a href="https://mcptoplist.com/server/io.github.Ikalus1988%2Fmisakanet"><img src="https://img.shields.io/badge/MCP-Toplist-blue" alt="MCP Toplist"></a>
|
|
53
|
+
<a href="https://dshfind.com/en/plugins/Ikalus1988/MisakaNet?ref=badge"><img src="https://dshfind.com/api/badge/Ikalus1988/MisakaNet?metric=downloads&lang=en" alt="DSH plugin downloads on dshfind"></a>
|
|
53
54
|
</p>
|
|
54
55
|
|
|
55
56
|
---
|
|
@@ -63,11 +64,13 @@ account or a token.
|
|
|
63
64
|
|---|---|
|
|
64
65
|
| **DeepSeek Harness** | `dsh plugin --profile web add misakanet` — or type `misakanet` in **Add plugin** |
|
|
65
66
|
| **Claude Code** | `/plugin marketplace add Ikalus1988/MisakaNet` then `/plugin install misakanet@misakanet` |
|
|
66
|
-
| Installer-managed | Claude Code · Codex · Hermes · OpenClaw · codewhale · Cursor · Gemini CLI · Copilot CLI · OpenCode · Kiro · MiniMax Code |
|
|
67
|
+
| Installer-managed | Claude Code · Codex · Hermes · OpenClaw · codewhale · Cursor · Gemini CLI · Copilot CLI · OpenCode · Kiro · MiniMax Code · DeepSeek Harness |
|
|
67
68
|
| **Any other MCP client** | point it at `https://misakanet.org/mcp` — public endpoint, anonymous reads |
|
|
68
69
|
| **Your own code** | `pip install misakanet-core` (library) · `pip install misakanet` (stdio server) |
|
|
69
70
|
|
|
70
71
|
The installer-managed row is what `npx @misaka-net/misakanet-setup` writes into each host's config.
|
|
72
|
+
For DeepSeek Harness that is the **rules half only** (`$DSH_HOME/AGENTS.md`): its tools come from the
|
|
73
|
+
plugin in the row above, which the installer prints as the one manual step.
|
|
71
74
|
|
|
72
75
|
Updates: `dsh plugin --profile web update misakanet@latest`.
|
|
73
76
|
Installer updates: `npx @misaka-net/misakanet-setup@latest`.
|
|
@@ -99,7 +102,9 @@ applies a fix somebody already verified → if nothing matches, the miss becomes
|
|
|
99
102
|
intake becomes a lesson for the next agent.
|
|
100
103
|
|
|
101
104
|
Every lesson is a Markdown file: reviewed like code, graded by evidence, retrieved with BM25 over the
|
|
102
|
-
Python standard library.
|
|
105
|
+
Python standard library. The hosted MCP endpoint adds an **optional semantic fallback** (bge-m3 +
|
|
106
|
+
Vectorize) only when BM25 returns no match; `MISAKANET_SEMANTIC=0` disables it. No vector database is
|
|
107
|
+
required locally, and no server unless you want one.
|
|
103
108
|
|
|
104
109
|
| | |
|
|
105
110
|
|---|---|
|
|
@@ -111,9 +116,9 @@ Python standard library. No vector database, no embedding model, no server unles
|
|
|
111
116
|
|---|---|
|
|
112
117
|
| ❌ a general memory system | ✅ a failure-recovery knowledge layer |
|
|
113
118
|
| ❌ an agent runtime | ✅ a searchable lesson database |
|
|
114
|
-
| ❌ a vector database / RAG stack | ✅ BM25
|
|
119
|
+
| ❌ a mandatory vector database / RAG stack | ✅ BM25 first; optional, disable-able semantic fallback on the hosted endpoint |
|
|
115
120
|
| ❌ a skill marketplace | ✅ debugging knowledge from real sessions |
|
|
116
|
-
| ❌ a
|
|
121
|
+
| ❌ a general-purpose benchmark platform | ✅ a corpus graded by evidence; a 267-query retrieval run is published, and the older recitation benchmark is paused |
|
|
117
122
|
|
|
118
123
|
### What it answers — and what it does not
|
|
119
124
|
|
|
@@ -135,12 +140,55 @@ not to fail again. MisakaNet is only the second thing. → [FAQ](FAQ.md)
|
|
|
135
140
|
- **The top of the evidence ladder is empty, and this page says so.** `E4` needs an observation —
|
|
136
141
|
two distinct clients actually retrieving a lesson — and the corpus has none yet
|
|
137
142
|
([distribution](docs/trust-semantics.md), regenerated and checked by CI).
|
|
138
|
-
- **Search is BM25
|
|
139
|
-
|
|
140
|
-
`no_match` plus an intake
|
|
143
|
+
- **Search is BM25 first; the hosted endpoint can fall back to embeddings.** Local/offline remains
|
|
144
|
+
stdlib BM25; the fallback runs only when BM25 returns no match, and `MISAKANET_SEMANTIC=0` turns it
|
|
145
|
+
off. A corpus gap is still a miss: the reply is `no_match` plus an intake, not an empty page.
|
|
141
146
|
- **Reported gaps arrive faster than they are triaged.** The intake queue is the honest bottleneck.
|
|
147
|
+
- **The corpus is downloadable and citable.** `data.misakanet.org` publishes a versioned
|
|
148
|
+
bundle — the full corpus, the benchmark query set, and every run artifact — with a
|
|
149
|
+
per-file SHA-256 and an explicit license per file ([DATASET.md](DATASET.md)). The counts
|
|
150
|
+
live in each version's `manifest.json`, not in prose.
|
|
142
151
|
|
|
143
|
-
##
|
|
152
|
+
## Retrieval benchmark: 267 queries, BM25 vs semantic fallback
|
|
153
|
+
|
|
154
|
+
On 2026-10-27 the hosted endpoint's retrieval path was evaluated on **267 queries, one per lesson**
|
|
155
|
+
generated by Workers AI 70B. Both arms used the same eval worker and corpus; only
|
|
156
|
+
`MISAKANET_SEMANTIC` changed.
|
|
157
|
+
|
|
158
|
+
| arm | R@1 | R@5 | no_match |
|
|
159
|
+
|---|---|---|---|
|
|
160
|
+
| BM25 | 0.210 | 0.225 | 0.712 |
|
|
161
|
+
| semantic fallback | **0.446** | **0.472** | **0.453** |
|
|
162
|
+
|
|
163
|
+
The fallback improves R@1 by **23.6 points** and cuts `no_match` by **25.9 points**. It does **not**
|
|
164
|
+
meet the R@1 half of the target (`0.446 < 0.46`); `no_match` is within the target
|
|
165
|
+
(`0.453 ≤ 0.50`). A 113-query subset reaches 0.460/0.460, while 250 reaches 0.404/0.504: the sample is
|
|
166
|
+
large enough to show the fallback works, not large enough to call R@1 solved.
|
|
167
|
+
|
|
168
|
+
This is 267 queries from the 300-entry pool, not the full 650; 33 pool entries yielded no query, 2 semantic requests
|
|
169
|
+
errored, and the eval corpus was not refreshed. → [full report](docs/benchmarks/retrieval-l1-267-2026-10-27.md)
|
|
170
|
+
|
|
171
|
+
### What changed since: the gate, not the model
|
|
172
|
+
|
|
173
|
+
The table above measures a configuration production no longer uses. Until 2026-10-28 the semantic path
|
|
174
|
+
ran **only when BM25 returned nothing**, so a confident-but-wrong lexical hit suppressed it — 13 of the
|
|
175
|
+
20 recorded misses were exactly that. For queries containing CJK characters the two ranked lists are now
|
|
176
|
+
**fused** (reciprocal-rank) instead of one gating the other.
|
|
177
|
+
|
|
178
|
+
Measured on the live endpoint, 64 Chinese queries, 0 transport failures:
|
|
179
|
+
|
|
180
|
+
| bucket | n | before | after |
|
|
181
|
+
|---|---|---|---|
|
|
182
|
+
| zh → zh | 31 | 0.710 | **0.774** |
|
|
183
|
+
| **zh → en** | 33 | 0.455 | **0.697** |
|
|
184
|
+
| zh → en, low overlap | 14 | 0.214 | **0.500** |
|
|
185
|
+
|
|
186
|
+
For scale: `bge-m3` pure cosine already reached 0.636 on the cross-lingual bucket, and a `gemini`
|
|
187
|
+
embedding 0.727 — so most of the gap was the decision to consult the vector path, not the model.
|
|
188
|
+
English queries are untouched (the fuse is gated on CJK characters) and sit at 15/20 against a floor of
|
|
189
|
+
16, which is where they were before this change. → [full report](docs/benchmarks/cjk-fusion-2026-10-28.md)
|
|
190
|
+
|
|
191
|
+
## Recitation benchmark: what it measured, and why it is paused
|
|
144
192
|
|
|
145
193
|
One narrow question: **when you hand a model a lesson, how much of that lesson comes back?** The
|
|
146
194
|
metric is `lesson_hit_rate` — the share of the injected lesson's commands reproduced in the answer.
|
|
@@ -208,14 +256,15 @@ command for every client. The ten supported clients, and what each one gets, are
|
|
|
208
256
|
[the install page](https://misakanet.org/install/). Verify an install with
|
|
209
257
|
`npx @misaka-net/misakanet-setup --verify`; undo it with `--uninstall`.
|
|
210
258
|
Reads are anonymous and unlimited; registration is for writing ([why](AGENTS.md#33-注册与配额)).
|
|
211
|
-
Node ≥ 18 for the installer, Python ≥ 3.
|
|
259
|
+
Node ≥ 18 for the installer, Python ≥ 3.11 for the library and stdio server. Python 3.10 reached end-of-life on 2026-10-01, so this release needs 3.11 or newer — upgrade the interpreter before upgrading the package.
|
|
212
260
|
|
|
213
261
|
### Other channels
|
|
214
262
|
|
|
215
263
|
- **Claude Code, as a plugin** — `/plugin marketplace add Ikalus1988/MisakaNet`, then
|
|
216
264
|
`/plugin install misakanet@misakanet`: the hosted MCP tools, no installer, no local process.
|
|
217
265
|
- **DeepSeek Harness** — `dsh plugin --profile web add misakanet`: `mcp__misakanet__*` tools plus the
|
|
218
|
-
session panel.
|
|
266
|
+
session panel. The rules half (the "search before you act" block) is written by the installer into
|
|
267
|
+
`$DSH_HOME/AGENTS.md`; dsh has no plugin extension point for it, so the two halves stay separate.
|
|
219
268
|
- **Call the endpoint yourself** — the `curl` in [Your first five minutes](#your-first-five-minutes).
|
|
220
269
|
- **The library in your own code** — `pip install misakanet-core`.
|
|
221
270
|
|
|
@@ -227,7 +276,7 @@ Node ≥ 18 for the installer, Python ≥ 3.10 for the library and stdio server.
|
|
|
227
276
|
- This repository (git) is also a **Claude Code plugin marketplace** —
|
|
228
277
|
`/plugin marketplace add Ikalus1988/MisakaNet`.
|
|
229
278
|
- `misakanet` (PyPI) is the stdio **MCP server** — `python3 -m misakanet.server`.
|
|
230
|
-
- `misakanet-core` (PyPI) is the **library** (stdlib-only BM25) —
|
|
279
|
+
- `misakanet-core` (PyPI) is the **library** (stdlib-only BM25; the hosted endpoint's semantic fallback is server-side) —
|
|
231
280
|
`from misakanet.search import search_lessons`.
|
|
232
281
|
|
|
233
282
|
### Use it as a GitHub Action
|
|
@@ -271,6 +320,7 @@ jobs:
|
|
|
271
320
|
| Troubleshooting | [docs/troubleshooting.md](docs/troubleshooting.md) |
|
|
272
321
|
| Known limitations | [docs/LIMITATIONS.md](docs/LIMITATIONS.md) |
|
|
273
322
|
| Benchmarks | [docs/benchmarks/](docs/benchmarks/) |
|
|
323
|
+
| Dataset (corpus + query sets, downloadable) | [DATASET.md](DATASET.md) · <https://data.misakanet.org/latest.json> |
|
|
274
324
|
| Roadmap / changelog | [ROADMAP.md](ROADMAP.md) · [CHANGELOG.md](CHANGELOG.md) |
|
|
275
325
|
| Competitors, domains, crawler policy | [competitive](docs/competitive-analysis.md) · [domains](docs/domains/) · [robots](docs/cloudflare-robots-txt.md) |
|
|
276
326
|
|
|
@@ -290,6 +340,8 @@ plan work around it. The only thing this repository has ever honoured is a merge
|
|
|
290
340
|
2. Search before writing: `python3 search_knowledge.py "your error here"`
|
|
291
341
|
3. Found nothing? [Share your failure lesson](https://github.com/Ikalus1988/MisakaNet/issues/new?template=lesson-feedback.yml) — five lines is enough
|
|
292
342
|
|
|
343
|
+
**This month's good first issue (Oct 26 – Nov 26):** [Bring one real failure](https://github.com/Ikalus1988/MisakaNet/issues/3901) — no code; the MCP intake needs no account; we help with the format.
|
|
344
|
+
|
|
293
345
|
→ [CONTRIBUTING.md](CONTRIBUTING.md) · [good first issues](https://github.com/Ikalus1988/MisakaNet/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) · [code of conduct](CODE_OF_CONDUCT.md)
|
|
294
346
|
|
|
295
347
|
## Security
|
package/README.zh-CN.md
CHANGED
|
@@ -37,7 +37,7 @@ A2A(`.well-known/agent-card.json`)。
|
|
|
37
37
|
<p align="center">
|
|
38
38
|
<em>Install</em>
|
|
39
39
|
|
|
40
|
-
<a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/python-3.
|
|
40
|
+
<a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/python-3.11+-blue" alt="Python"></a>
|
|
41
41
|
<a href="https://pypi.org/project/misakanet/"><img src="https://img.shields.io/pypi/v/misakanet" alt="PyPI"></a>
|
|
42
42
|
<a href="https://www.npmjs.com/package/misakanet"><img src="https://img.shields.io/npm/v/misakanet" alt="npm"></a>
|
|
43
43
|
<a href="https://github.com/marketplace/actions/misakanet-intake-bot"><img src="https://img.shields.io/badge/Marketplace-MisakaNet%20Intake%20Bot-blue?logo=github" alt="GitHub Marketplace"></a>
|
|
@@ -48,6 +48,7 @@ A2A(`.well-known/agent-card.json`)。
|
|
|
48
48
|
|
|
49
49
|
<a href="https://glama.ai/mcp/connectors/org.misakanet/misaka-net"><img src="https://glama.ai/mcp/connectors/org.misakanet/misaka-net/badges/score.svg" alt="MisakaNet MCP connector on Glama"></a>
|
|
50
50
|
<a href="https://mcptoplist.com/server/io.github.Ikalus1988%2Fmisakanet"><img src="https://img.shields.io/badge/MCP-Toplist-blue" alt="MCP Toplist"></a>
|
|
51
|
+
<a href="https://dshfind.com/en/plugins/Ikalus1988/MisakaNet?ref=badge"><img src="https://dshfind.com/api/badge/Ikalus1988/MisakaNet?metric=downloads&lang=zh" alt="DSH 插件下载量(dshfind)"></a>
|
|
51
52
|
</p>
|
|
52
53
|
|
|
53
54
|
---
|
|
@@ -104,9 +105,9 @@ curl -sS https://misakanet.org/mcp \
|
|
|
104
105
|
|---|---|
|
|
105
106
|
| ❌ 通用记忆系统 | ✅ 失败修复知识层 |
|
|
106
107
|
| ❌ Agent 运行时 | ✅ 可检索的 lesson 数据库 |
|
|
107
|
-
| ❌ 向量数据库 / RAG 栈 | ✅ BM25 关键词检索,纯标准库(仍需 Python ≥ 3.
|
|
108
|
+
| ❌ 向量数据库 / RAG 栈 | ✅ BM25 关键词检索,纯标准库(仍需 Python ≥ 3.11) |
|
|
108
109
|
| ❌ 技能市场 | ✅ 来自真实会话的排错知识 |
|
|
109
|
-
| ❌
|
|
110
|
+
| ❌ 通用基准平台 | ✅ 按证据分级语料;公开了 267 条检索评测,旧的复述基准已暂停 |
|
|
110
111
|
|
|
111
112
|
### 它能回答什么、不能回答什么
|
|
112
113
|
|
|
@@ -130,7 +131,24 @@ MisakaNet 只回答它已经索引过的失败。没有命中时,返回的是
|
|
|
130
131
|
所以未命中返回 `no_match` 加 intake,而不是空白页。
|
|
131
132
|
- **报料速度高于分诊速度**:intake 队列是诚实的瓶颈。
|
|
132
133
|
|
|
133
|
-
##
|
|
134
|
+
## 检索评测:267 条,BM25 vs semantic fallback
|
|
135
|
+
|
|
136
|
+
2026-10-27 在 hosted endpoint 上评测了 **267 条** query,每篇 lesson 一条,由 Workers AI 70B 生成。
|
|
137
|
+
两个 arm 用同一个 eval worker 和 corpus,只切换 `MISAKANET_SEMANTIC`。
|
|
138
|
+
|
|
139
|
+
| arm | R@1 | R@5 | no_match |
|
|
140
|
+
|---|---|---|---|
|
|
141
|
+
| BM25 | 0.210 | 0.225 | 0.712 |
|
|
142
|
+
| semantic fallback | **0.446** | **0.472** | **0.453** |
|
|
143
|
+
|
|
144
|
+
semantic fallback 把 R@1 提高 **23.6 个百分点**,把 `no_match` 降低 **25.9 个百分点**。但 R@1
|
|
145
|
+
**0.446 < 0.46**,没有过 R@1 条件;`no_match` **0.453 ≤ 0.50**,过了 no_match 条件。113 条子集是
|
|
146
|
+
0.460/0.460,250 条是 0.404/0.504——样本足以说明 fallback 有效,不足以宣布 R@1 已解决。
|
|
147
|
+
|
|
148
|
+
这是 300 篇里的 267 条,不是完整 650;33 篇没有生成出 query,2 个 semantic 请求报错,eval corpus
|
|
149
|
+
也没有刷新。→ [完整报告](docs/benchmarks/retrieval-l1-267-2026-10-27.md)
|
|
150
|
+
|
|
151
|
+
## 复述基准:它测了什么,为什么暂停了
|
|
134
152
|
|
|
135
153
|
只测了一个很窄的问题:**把一条 lesson 塞给模型,它会复述回多少?** 指标是 `lesson_hit_rate`——被注入
|
|
136
154
|
lesson 里的命令有多大概率出现在回答里。读数字前先读设定:本基准的场景就是**每篇 lesson 自己的标题**,
|
|
@@ -212,7 +230,7 @@ with_lesson hit rate (per run date)
|
|
|
212
230
|
`misakanet_write_lesson` 与 `misakanet_preflight`([原因](AGENTS.md#33-注册与配额))。
|
|
213
231
|
|
|
214
232
|
验证安装:`npx @misaka-net/misakanet-setup --verify`;卸载:`--uninstall`。
|
|
215
|
-
前置:安装器要 Node ≥ 18,库与 stdio server 要 Python ≥ 3.10
|
|
233
|
+
前置:安装器要 Node ≥ 18,库与 stdio server 要 Python ≥ 3.11。Python 3.10 已于 2026-10-01 结束支持(EOL),所以本版本需要 3.11 或更新——升级本包之前先升级解释器。
|
|
216
234
|
|
|
217
235
|
### 作为 GitHub Action 使用
|
|
218
236
|
|
package/lib/client.js
CHANGED
|
@@ -193,6 +193,12 @@ window.__ModuleLoader__.load({
|
|
|
193
193
|
"panel.asked.voteTwo": "2 human confirmations — recorded as a hint; the E4 rung comes from observable reuse, not a vote.",
|
|
194
194
|
"panel.asked.voteMore": "human confirmations — recorded as a hint; the E4 rung comes from observable reuse, not a vote.",
|
|
195
195
|
"panel.asked.noMatchTag": "no match",
|
|
196
|
+
// A degraded read is not a miss and not a zero: the server answered with `degraded: true`,
|
|
197
|
+
// `degraded_reasons` and a one-line `notice`, and the row has to say so instead of printing a
|
|
198
|
+
// count the corpus behind it could not stand behind (issue #3511).
|
|
199
|
+
"panel.asked.degradedTpl": "results may be incomplete ({reason})",
|
|
200
|
+
"panel.asked.degradedTag": "degraded",
|
|
201
|
+
"panel.asked.degradedWhy": "This read was degraded — it is not proof that no lesson exists.",
|
|
196
202
|
"panel.asked.agentNext": "Hand this line to the agent:",
|
|
197
203
|
"panel.asked.agentNextCopy": "Copy the next step",
|
|
198
204
|
"panel.asked.agentNextText": "Please record this gap with the MisakaNet intake so the next person finds it: misakanet_submit_intake problem: ",
|
|
@@ -277,6 +283,7 @@ window.__ModuleLoader__.load({
|
|
|
277
283
|
// and splitting the phrase across two slots later made it read `1 篇 篇被课程复用`.
|
|
278
284
|
"panel.strip.hitTpl": "{count} hit · ",
|
|
279
285
|
"panel.strip.missTpl": "{count} found nothing · ",
|
|
286
|
+
"panel.strip.degradedTpl": "{count} degraded · ",
|
|
280
287
|
"panel.strip.surfacedTpl": "{count} surfaced · ",
|
|
281
288
|
"panel.strip.helpfulTpl": "({n} helpful) · ",
|
|
282
289
|
"panel.strip.convertedTpl": "({n} became a lesson)",
|
|
@@ -359,6 +366,9 @@ window.__ModuleLoader__.load({
|
|
|
359
366
|
"panel.asked.voteTwo": "2 个人工确认 —— 只记为提示,不算证据:E4 那一档来自可观测的复用,投票给不了。",
|
|
360
367
|
"panel.asked.voteMore": "个人工确认 —— 只记为提示,不算证据:E4 那一档来自可观测的复用,投票给不了。",
|
|
361
368
|
"panel.asked.noMatchTag": "无命中",
|
|
369
|
+
"panel.asked.degradedTpl": "结果可能不完整({reason})",
|
|
370
|
+
"panel.asked.degradedTag": "降级读取",
|
|
371
|
+
"panel.asked.degradedWhy": "这是一次降级读取 —— 不代表真的没有这条课程。",
|
|
362
372
|
"panel.asked.agentNext": "把这行交给 agent:",
|
|
363
373
|
"panel.asked.agentNextCopy": "复制下一步",
|
|
364
374
|
"panel.asked.agentNextText": "请用 MisakaNet 报料记下这个缺口,让下一个人能搜到:misakanet_submit_intake problem: ",
|
|
@@ -427,6 +437,7 @@ window.__ModuleLoader__.load({
|
|
|
427
437
|
"settings.what.row": "条目",
|
|
428
438
|
"panel.strip.hitTpl": "{n} {unit}命中过课程 · ",
|
|
429
439
|
"panel.strip.missTpl": "{n} {unit}没命中 · ",
|
|
440
|
+
"panel.strip.degradedTpl": "{n} {unit}降级读取 · ",
|
|
430
441
|
"panel.strip.surfacedTpl": "{n} {unit}命中过课程 · ",
|
|
431
442
|
"panel.strip.helpfulTpl": "({n} 有用) · ",
|
|
432
443
|
"panel.strip.convertedTpl": "({n} 转化为课程)",
|
|
@@ -902,6 +913,11 @@ var DEFAULT_MCP_URL = "https://misakanet.org/mcp";
|
|
|
902
913
|
query: String(event.query || payload.query || ""),
|
|
903
914
|
hits: results.length,
|
|
904
915
|
noMatch: Boolean(payload.no_match),
|
|
916
|
+
// The server's own degradation report (issue #3512), kept with the sighting so every
|
|
917
|
+
// surface that renders this entry can say "incomplete" instead of "zero".
|
|
918
|
+
degraded: Boolean(payload.degraded),
|
|
919
|
+
degradedReasons: Array.isArray(payload.degraded_reasons) ? payload.degraded_reasons.slice() : [],
|
|
920
|
+
notice: typeof payload.notice === "string" ? payload.notice : "",
|
|
905
921
|
lessonId: top && top.id ? String(top.id) : "",
|
|
906
922
|
title: top ? String(top.title || top.id || "") : "",
|
|
907
923
|
evidence: top ? String(top.evidence_level || "") : "",
|
|
@@ -1078,6 +1094,20 @@ var DEFAULT_MCP_URL = "https://misakanet.org/mcp";
|
|
|
1078
1094
|
return n + " " + (n === 1 ? singular : plural);
|
|
1079
1095
|
}
|
|
1080
1096
|
|
|
1097
|
+
/**
|
|
1098
|
+
* The reason a degraded read is degraded: the server's own stable reason ids, joined — the field
|
|
1099
|
+
* names are the branch points `searchDegraded` publishes, so the label names what failed rather
|
|
1100
|
+
* than paraphrasing it. A payload that claims `degraded` without reasons falls back to the tag
|
|
1101
|
+
* rather than printing an empty parenthesis.
|
|
1102
|
+
*/
|
|
1103
|
+
function degradedReason(payload) {
|
|
1104
|
+
// Two shapes, one meaning: the wire payload (`degraded_reasons`) and the recorded session
|
|
1105
|
+
// entry (`degradedReasons`) hold the same ids.
|
|
1106
|
+
var reasons = payload && (Array.isArray(payload.degraded_reasons) ? payload.degraded_reasons
|
|
1107
|
+
: (Array.isArray(payload.degradedReasons) ? payload.degradedReasons : []));
|
|
1108
|
+
return reasons.length ? reasons.join(", ") : T("panel.asked.degradedTag");
|
|
1109
|
+
}
|
|
1110
|
+
|
|
1081
1111
|
/** mm:ss for today, else the date — the event's own time, never the render time. */
|
|
1082
1112
|
function clockOf(at) {
|
|
1083
1113
|
try {
|
|
@@ -1156,6 +1186,16 @@ var DEFAULT_MCP_URL = "https://misakanet.org/mcp";
|
|
|
1156
1186
|
head.push(h("span", { key: "state", style: DIM }, " " + T("panel.asked.searching")));
|
|
1157
1187
|
} else if (phase === "start") {
|
|
1158
1188
|
head.push(h("span", { key: "state" }, " " + (query || T("panel.asked.searching"))));
|
|
1189
|
+
} else if (payload && payload.degraded) {
|
|
1190
|
+
// A degraded read renders as the reason plus the server's one-line explanation — never as
|
|
1191
|
+
// only a result count, and never as `no_match` ("no lesson matched"), which is the one
|
|
1192
|
+
// sentence a frozen corpus cannot earn (issue #3511, measured 2026-10-18).
|
|
1193
|
+
head.push(h("span", { key: "state" }, " " + T("panel.asked.degradedTpl",
|
|
1194
|
+
{ reason: degradedReason(payload) })));
|
|
1195
|
+
if (results.length) {
|
|
1196
|
+
head.push(h("span", { key: "hits", style: DIM }, " · " + count(results.length, T("unit.lesson"), T("unit.lessons"))));
|
|
1197
|
+
}
|
|
1198
|
+
head.push(h("span", { key: "notice", style: DIM }, " · " + (payload.notice || T("panel.asked.degradedWhy"))));
|
|
1159
1199
|
} else if (payload && payload.no_match) {
|
|
1160
1200
|
head.push(h("span", { key: "state" }, " " + T("panel.asked.noMatch")));
|
|
1161
1201
|
} else if (results.length) {
|
|
@@ -1379,7 +1419,10 @@ var DEFAULT_MCP_URL = "https://misakanet.org/mcp";
|
|
|
1379
1419
|
var lessons = log ? Object.keys(log.lessons).map(function (id) { return log.lessons[id]; }) : [];
|
|
1380
1420
|
var stats = browserStats();
|
|
1381
1421
|
var withLesson = searches.filter(function (s) { return s.lessonId; }).length;
|
|
1382
|
-
|
|
1422
|
+
// A degraded search is counted in its own bucket and *not* as "found nothing": that phrase is
|
|
1423
|
+
// the product's confident zero, and the server said the read could not earn it (issue #3511).
|
|
1424
|
+
var degradedCount = searches.filter(function (s) { return s.degraded; }).length;
|
|
1425
|
+
var noMatch = searches.filter(function (s) { return s.noMatch && !s.lessonId && !s.degraded; }).length;
|
|
1383
1426
|
var votes = log ? log.votes : [];
|
|
1384
1427
|
var helpful = votes.filter(function (v) { return v.kind === "helpful"; }).length;
|
|
1385
1428
|
var converted = intakes.filter(function (r) { return r.state === "converted"; }).length;
|
|
@@ -1517,6 +1560,9 @@ var DEFAULT_MCP_URL = "https://misakanet.org/mcp";
|
|
|
1517
1560
|
count(searches.length, T("unit.search"), T("unit.searches")) + " · " +
|
|
1518
1561
|
T("panel.strip.hitTpl", { count: count(withLesson, T("unit.lesson"), T("unit.lessons")), n: withLesson, unit: T("unit.lessons") }) +
|
|
1519
1562
|
T("panel.strip.missTpl", { count: count(noMatch, T("unit.lesson"), T("unit.lessons")), n: noMatch, unit: T("unit.lessons") }) +
|
|
1563
|
+
// …and the degraded bucket says so out loud, so a degraded search is never counted as a
|
|
1564
|
+
// normal one without a label of its own.
|
|
1565
|
+
T("panel.strip.degradedTpl", { count: count(degradedCount, T("unit.search"), T("unit.searches")), n: degradedCount, unit: T("unit.searches") }) +
|
|
1520
1566
|
T("panel.strip.surfacedTpl", { count: count(lessons.length, T("unit.lesson"), T("unit.lessons")), n: lessons.length, unit: T("unit.lessons") }) +
|
|
1521
1567
|
count(votes.length, T("unit.vote"), T("unit.votes")) + " " + T("panel.strip.helpfulTpl", { count: count(helpful, T("unit.vote"), T("unit.votes")), n: helpful, unit: T("unit.votes") }) +
|
|
1522
1568
|
count(intakes.length, T("unit.report"), T("unit.reports")) + " " + T("panel.strip.convertedTpl", { count: count(converted, T("unit.report"), T("unit.reports")), n: converted, unit: T("unit.reports") })));
|
|
@@ -1614,7 +1660,11 @@ var DEFAULT_MCP_URL = "https://misakanet.org/mcp";
|
|
|
1614
1660
|
? (entry.voteKind === "helpful" ? T("panel.asked.voteHelpful") : T("panel.asked.voteNotHelpful"))
|
|
1615
1661
|
: (entry.lessonId ? T("panel.asked.voteNone") : T("panel.asked.voteNoLesson"));
|
|
1616
1662
|
var said = panelVotes[key];
|
|
1617
|
-
|
|
1663
|
+
// A degraded sighting is not a miss: "found nothing" is exactly the claim a frozen
|
|
1664
|
+
// corpus cannot make, and offering the intake line from one would file a gap for a
|
|
1665
|
+
// lesson that may well exist (issue #3511).
|
|
1666
|
+
var isDegraded = Boolean(entry.degraded);
|
|
1667
|
+
var missed = !isDegraded && (Boolean(entry.noMatch) || !entry.lessonId);
|
|
1618
1668
|
var controls = [];
|
|
1619
1669
|
if (entry.lessonId && !entry.voted) {
|
|
1620
1670
|
controls.push(h("button", {
|
|
@@ -1643,9 +1693,14 @@ var DEFAULT_MCP_URL = "https://misakanet.org/mcp";
|
|
|
1643
1693
|
h("div", { key: "head", style: { borderLeft: "2px solid " + (missed ? "rgba(127,127,127,0.9)" : "transparent"), paddingLeft: missed ? "6px" : "8px" } },
|
|
1644
1694
|
h("span", { style: DIM }, clockOf(entry.at) + " "),
|
|
1645
1695
|
h("span", {}, entry.query || T("panel.asked.noQuery")),
|
|
1646
|
-
h("span", { style: DIM }, " " + (
|
|
1696
|
+
h("span", { style: DIM }, " " + (isDegraded
|
|
1697
|
+
? T("panel.asked.degradedTpl", { reason: degradedReason(entry) })
|
|
1698
|
+
+ (entry.hits ? " · " + count(entry.hits, T("unit.lesson")) : "")
|
|
1699
|
+
: (entry.noMatch ? T("panel.asked.nothingFound") : count(entry.hits, T("unit.lesson"))))),
|
|
1647
1700
|
missed ? h("span", { style: Object.assign({}, DIM, { border: "1px solid " + MUTED, borderRadius: "4px", padding: "0 4px", marginLeft: "4px", fontSize: "11px" }) },
|
|
1648
1701
|
T("panel.asked.noMatchTag")) : null,
|
|
1702
|
+
isDegraded ? h("span", { style: Object.assign({}, DIM, { border: "1px solid " + MUTED, borderRadius: "4px", padding: "0 4px", marginLeft: "4px", fontSize: "11px" }) },
|
|
1703
|
+
T("panel.asked.degradedTag")) : null,
|
|
1649
1704
|
entry.title ? h("span", { style: DIM }, " · top: " + entry.title + (entry.evidence ? " (" + entry.evidence + ")" : "")) : null,
|
|
1650
1705
|
entry.lessonId ? h("span", { style: DIM }, " · " + vote) : null),
|
|
1651
1706
|
];
|
|
@@ -1655,6 +1710,10 @@ var DEFAULT_MCP_URL = "https://misakanet.org/mcp";
|
|
|
1655
1710
|
h("code", { style: { fontSize: "11px" } }, intakeNextStep(entry.query))));
|
|
1656
1711
|
lines.push(h("div", { key: "why", style: DIM }, T("panel.asked.agentNextWhy")));
|
|
1657
1712
|
}
|
|
1713
|
+
if (isDegraded) {
|
|
1714
|
+
// The one-line explanation, from the server's own `notice` when it sent one.
|
|
1715
|
+
lines.push(h("div", { key: "why", style: DIM }, entry.notice || T("panel.asked.degradedWhy")));
|
|
1716
|
+
}
|
|
1658
1717
|
if (controls.length || said) {
|
|
1659
1718
|
lines.push(h("div", { key: "do" }, controls,
|
|
1660
1719
|
said ? h("span", { style: DIM }, said) : null));
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "misakanet",
|
|
3
|
-
"version": "
|
|
3
|
+
"version": "3.0.0",
|
|
4
4
|
"license": "Apache-2.0",
|
|
5
5
|
"description": "MisakaNet failure-memory skill — evidence-rated failure lessons for AI agents. DSH bundle: registers the MCP tools as mcp__misakanet__* (misakanet_search/get_lesson/…), served by the public Streamable HTTP endpoint https://misakanet.org/mcp — live tools from an npm install, in any profile, with no local python. A profile that prefers the repo's own stdio server can override the bundle row.",
|
|
6
6
|
"icon": "icon.svg",
|
|
@@ -80,7 +80,7 @@
|
|
|
80
80
|
"preview": "wrangler dev"
|
|
81
81
|
},
|
|
82
82
|
"devDependencies": {
|
|
83
|
-
"wrangler": "^4.
|
|
83
|
+
"wrangler": "^4.152.0"
|
|
84
84
|
},
|
|
85
85
|
"overrides": {
|
|
86
86
|
"sharp": "0.35.5",
|