pi-jev-wiki 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +21 -0
- package/README.md +14 -4
- package/package.json +1 -1
- package/skills/llm-wiki/SKILL.md +107 -10
- package/src/config.ts +2 -1
- package/src/extension.ts +25 -1
- package/src/pipeline/adjudicate.ts +14 -0
- package/src/triage.ts +190 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,10 +1,31 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.3.0 — 2026-09-20
|
|
4
|
+
|
|
5
|
+
### Added
|
|
6
|
+
|
|
7
|
+
- `wiki_triage`: rejected-claim triage. Lists recent rejections with Jev scores, whether the
|
|
8
|
+
problem is evidence or policy, a concrete remedy per claim, and the accepted-vs-rejected
|
|
9
|
+
derivability ranges so miscalibrated thresholds are visible.
|
|
10
|
+
|
|
11
|
+
### Changed
|
|
12
|
+
|
|
13
|
+
- The `llm-wiki` skill now teaches the judgment criteria comprehensively: every score, the decision
|
|
14
|
+
gates in order, the framing exception, which evidence kinds pass, phrasing rules, a pre-submission
|
|
15
|
+
checklist, and threshold calibration.
|
|
16
|
+
- High-importance architecture, invariant, and decision framing that Jev rates derivable is queued
|
|
17
|
+
for confirmation instead of rejected (`thresholds.framingImportance`, default 0.6).
|
|
18
|
+
- Rejection guidance treats a correctly rejected claim as a working system rather than a failure;
|
|
19
|
+
`wiki_insights` guidelines point at the criteria and at `wiki_triage`.
|
|
20
|
+
|
|
3
21
|
## 0.2.0 — 2026-09-19
|
|
4
22
|
|
|
5
23
|
First public release. A pi package that builds and maintains a project mental-model wiki,
|
|
6
24
|
with [Jev](https://typesafe.ai) (TypeSafe System One) as the calibrated decision layer.
|
|
7
25
|
|
|
26
|
+
Published to npm as `pi-jev-wiki@0.2.0` on 2026-09-20 via an interactive publish, so this version
|
|
27
|
+
has no provenance attestation; the next CI-published release will carry provenance.
|
|
28
|
+
|
|
8
29
|
### Intake
|
|
9
30
|
|
|
10
31
|
- Research ingest (`wiki_ingest`): immutable raw sources, claim extraction with verbatim quote
|
package/README.md
CHANGED
|
@@ -146,7 +146,17 @@ under `docs/` and `research/` stays in the repository.
|
|
|
146
146
|
|
|
147
147
|
## Status
|
|
148
148
|
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
149
|
+
**Published:** [`pi-jev-wiki@0.2.0`](https://www.npmjs.com/package/pi-jev-wiki) — first release was
|
|
150
|
+
an interactive publish, so it has no provenance attestation; CI publishing is wired and the next
|
|
151
|
+
version bump will be the first release with provenance. Listed on the
|
|
152
|
+
[pi package gallery](https://pi.dev/packages/pi-jev-wiki).
|
|
153
|
+
|
|
154
|
+
Implemented through P3: both intake channels (research ingest + agent insights), architecture-first
|
|
155
|
+
pages, TOC/log, decision ledger, change-driven invalidation (`wiki_sync`), agent-managed review,
|
|
156
|
+
draft/auto writers with adaptive risk, pluggable search (index/BM25/qmd), hierarchical TOC,
|
|
157
|
+
lint/consolidation checks, redaction, cross-process locking, `wiki_doctor`/`wiki_setup`, and
|
|
158
|
+
offline unit + scale tests.
|
|
159
|
+
|
|
160
|
+
Not yet done: trusted publishing (OIDC) migration, the decision-quality evaluation harness, and the
|
|
161
|
+
remaining hardening items tracked in [`docs/HARDENING.md`](docs/HARDENING.md). See
|
|
162
|
+
[`docs/RELEASING.md`](docs/RELEASING.md) for the release process.
|
package/package.json
CHANGED
package/skills/llm-wiki/SKILL.md
CHANGED
|
@@ -10,6 +10,9 @@ the system: how it is structured, why, and what a change touches. It is not a co
|
|
|
10
10
|
Jev (a calibrated decision model) judges whether claims are grounded, derivable, durable, and
|
|
11
11
|
where they belong; you do the writing.
|
|
12
12
|
|
|
13
|
+
**Read [How claims are judged](#how-claims-are-judged) before composing insights.** Most
|
|
14
|
+
rejections are phrasing or evidence problems that are easy to avoid once you know the gates.
|
|
15
|
+
|
|
13
16
|
## Setup (when no key is configured)
|
|
14
17
|
|
|
15
18
|
If Jev calls fail with an authentication error, or the user asks how to connect a provider:
|
|
@@ -23,17 +26,94 @@ If Jev calls fail with an authentication error, or the user asks how to connect
|
|
|
23
26
|
Never print the key value. TypeSafe uses `TYPESAFE_API_KEY` (`JEV_TOKEN` also works); OpenRouter
|
|
24
27
|
uses `OPENROUTER_API_KEY` with provider `openrouter`, or pi's own `/login openrouter` credential.
|
|
25
28
|
|
|
26
|
-
##
|
|
29
|
+
## How claims are judged
|
|
30
|
+
|
|
31
|
+
You are the writer; Jev decides. These are the criteria, in the order they are applied.
|
|
32
|
+
|
|
33
|
+
### The quality bar
|
|
27
34
|
|
|
28
35
|
File only what the repository cannot answer cheaply:
|
|
29
36
|
|
|
30
|
-
- **Delete test** — if deleting the page would
|
|
31
|
-
does not belong.
|
|
37
|
+
- **Delete test** — if deleting the page would send a future agent straight back to reading code,
|
|
38
|
+
it does not belong. The one exception is high-importance framing (see the framing gate).
|
|
32
39
|
- **Decision test** — does it help decide *where a change belongs* or *what a change breaks*?
|
|
33
40
|
|
|
34
41
|
Good: module responsibilities and boundaries, dependency direction, data flow, invariants,
|
|
35
42
|
decisions with rationale, change-impact knowledge, historical attempts, external constraints,
|
|
36
|
-
domain glossary
|
|
43
|
+
domain glossary, gotchas that cost real debugging time.
|
|
44
|
+
|
|
45
|
+
Bad: function bodies, obvious implementation, transient task state, code excerpts, restated docs.
|
|
46
|
+
|
|
47
|
+
### What Jev scores
|
|
48
|
+
|
|
49
|
+
Every claim is judged against the evidence you attach, and returns calibrated scores:
|
|
50
|
+
|
|
51
|
+
| Score | Question |
|
|
52
|
+
|---|---|
|
|
53
|
+
| `grounded` | Does the evidence state or directly imply the claim? |
|
|
54
|
+
| `derivable` | Could a developer re-derive this from the repository in under a minute? |
|
|
55
|
+
| `durable` | Will it still be true and useful in a month? |
|
|
56
|
+
| `importance` | How load-bearing is it? (ephemeral → contextual → durable → canonical) |
|
|
57
|
+
| `criticality` | How costly is acting on it incorrectly? |
|
|
58
|
+
| `already_known` | Does the wiki already hold it? |
|
|
59
|
+
| `trustTier` | Basis of the claim: `source_document`, `verified_in_repo`, `user_stated`, `inference`, `speculation` |
|
|
60
|
+
|
|
61
|
+
### The decision gates, in order
|
|
62
|
+
|
|
63
|
+
| # | Condition | Outcome |
|
|
64
|
+
|---|---|---|
|
|
65
|
+
| 1 | `sensitive` ≥ 0.9 | **rejected** — secrets and PII are never filed |
|
|
66
|
+
| 2 | `derivable` ≥ 0.5 | **rejected** as implementation detail, *except* high-importance framing (below) |
|
|
67
|
+
| 3 | `already_known` ≥ 0.9 | **rejected** as duplicate — reinforce or extend the existing claim |
|
|
68
|
+
| 4 | `grounded` ≥ 0.8 and `importance` ≥ 1 | **filed**, or **reinforced** when it extends or agrees with existing claims |
|
|
69
|
+
| 5 | `grounded` ≥ 0.7 | **queued for review** (below auto-accept) |
|
|
70
|
+
| 6 | `grounded` < 0.7 | filed only with trust tier `user_stated` (stored with lower status) or `verified_in_repo`; `inference` and `speculation` never file |
|
|
71
|
+
| 7 | otherwise | **rejected** as unsupported |
|
|
72
|
+
|
|
73
|
+
**Framing gate (the exception to #2):** architecture, invariant, and decision claims with
|
|
74
|
+
`importance` ≥ 0.6 are **queued for confirmation** when Jev rates them derivable, instead of being
|
|
75
|
+
dropped. The reason: a future agent needs the frame even when the code technically contains the
|
|
76
|
+
pieces. Lower-importance derivable claims still fail the delete test.
|
|
77
|
+
|
|
78
|
+
### Evidence that passes
|
|
79
|
+
|
|
80
|
+
Attach evidence that *states* what you claim — Jev reads it.
|
|
81
|
+
|
|
82
|
+
| Kind | Use it for | Notes |
|
|
83
|
+
|---|---|---|
|
|
84
|
+
| `commit` | decisions and rationale | The strongest artifact for *why*. Use the hash of the commit that introduced the change; its message is read. |
|
|
85
|
+
| `source` | external knowledge, docs | Include a verbatim quote. Paraphrases fail grounding. |
|
|
86
|
+
| `user` | policies the user stated | Filed with `status: user-stated` (lower trust), not verified. |
|
|
87
|
+
| `file` | code-visible structure | The extension reads and excerpts the file for Jev. A bare path does not ground *intent* or *rationale* — pair it with a commit or a quote for "why" claims. |
|
|
88
|
+
| `command` / `test` | reproducible behavior | Include the command and the observed output. |
|
|
89
|
+
|
|
90
|
+
### Phrasing rules
|
|
91
|
+
|
|
92
|
+
- **Atomic**: one claim, one idea. Split mechanism, rationale, and consequence into separate claims.
|
|
93
|
+
- **Scope to the evidence**: if the quote supports X, do not claim X and Y.
|
|
94
|
+
- **Separate *what* from *why***: "the client retries on 429" is derivable → rejected; "retries
|
|
95
|
+
live in the client because provider gateways implement Retry-After inconsistently" is rationale → durable.
|
|
96
|
+
- **Prefer framing over detail**: decisions, invariants, boundaries, and gotchas over implementation facts.
|
|
97
|
+
- **Name the kind**: `decision`, `invariant`, `architecture`, `gotcha`, `pattern`, `procedure`,
|
|
98
|
+
`fact`, `preference` — it decides which gate applies.
|
|
99
|
+
|
|
100
|
+
### Pre-submission checklist
|
|
101
|
+
|
|
102
|
+
1. Can another agent re-derive it in a minute from the repo? → don't submit, or submit it as
|
|
103
|
+
high-importance framing.
|
|
104
|
+
2. Can I point to a quote, commit, or user sentence that states it? → if not, it will be rejected
|
|
105
|
+
as unsupported.
|
|
106
|
+
3. Is it transient task state? → don't submit.
|
|
107
|
+
4. Does the wiki already say it? → reinforce or extend the existing page instead.
|
|
108
|
+
5. Is it code-visible but load-bearing framing? → submit as architecture/invariant/decision; it
|
|
109
|
+
will be queued rather than dropped.
|
|
110
|
+
|
|
111
|
+
### Thresholds and calibration
|
|
112
|
+
|
|
113
|
+
Defaults: `autoAccept` 0.8, `minSupport` 0.7, `minDerivable` 0.5, `framingImportance` 0.6,
|
|
114
|
+
`minImportance` 1. They are configurable in `.pi/jev-wiki.json` (project) or
|
|
115
|
+
`~/.pi/agent/jev-wiki.json` (global). If rejections look wrong for this project, run
|
|
116
|
+
`wiki_triage`: it reports the accepted vs rejected score ranges and flags overlapping thresholds.
|
|
37
117
|
|
|
38
118
|
## Layout
|
|
39
119
|
|
|
@@ -69,6 +149,23 @@ Rejected claims are **not** written to the wiki — even when the user explicitl
|
|
|
69
149
|
content. Report what was rejected and why, so the user can supply better evidence or a stronger
|
|
70
150
|
artifact. Never hand-write a rejected claim into a page.
|
|
71
151
|
|
|
152
|
+
### When a claim is rejected
|
|
153
|
+
|
|
154
|
+
Rejections are diagnosable — run `wiki_triage` to see the scores, the reason, and the fix for each:
|
|
155
|
+
|
|
156
|
+
- **derivable from code**: add what the code cannot show — a commit message (evidence `kind: commit`),
|
|
157
|
+
a source quote, or the rationale behind the decision. High-importance architecture, invariant, and
|
|
158
|
+
decision framing is queued for confirmation instead of dropped.
|
|
159
|
+
- **unsupported**: attach evidence that *states* the claim: a verbatim quote, the commit that
|
|
160
|
+
introduced it, or the user's own words (`kind: user`). A file reference without a supporting
|
|
161
|
+
passage is not enough.
|
|
162
|
+
- **duplicate**: find the page that covers it and reinforce or extend it instead of filing a new claim.
|
|
163
|
+
- **sensitive**: never file it; redact secrets and PII first.
|
|
164
|
+
|
|
165
|
+
If a rejected claim still matters after applying the remedy, re-submit it with the better evidence
|
|
166
|
+
rather than writing it by hand. If it does not survive re-submission, tell the user why — a
|
|
167
|
+
correctly rejected claim is a working system, not a failure.
|
|
168
|
+
|
|
72
169
|
### Ingest a document (research channel)
|
|
73
170
|
|
|
74
171
|
1. `wiki_ingest` with `path` (or `text`) — it stores the raw source, extracts claims, and returns
|
|
@@ -78,12 +175,12 @@ artifact. Never hand-write a rejected claim into a page.
|
|
|
78
175
|
|
|
79
176
|
### Capture session insights (work channel)
|
|
80
177
|
|
|
81
|
-
1. Compose atomic insights
|
|
82
|
-
|
|
83
|
-
2. `wiki_insights` with the list — Jev filters
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
178
|
+
1. Compose atomic insights against the [criteria](#how-claims-are-judged). For decisions, attach
|
|
179
|
+
the introducing commit; for policy, the user's words; for documents, a verbatim quote.
|
|
180
|
+
2. `wiki_insights` with the list — Jev filters, relates them to existing knowledge, and chooses
|
|
181
|
+
placement. File evidence is read and excerpted for Jev, which grounds the decision.
|
|
182
|
+
3. Write or merge the **accepted** pages, then `wiki_finalize`. If a claim is queued under the
|
|
183
|
+
framing gate, it is worth filing: confirm it via `wiki_review` and write the page.
|
|
87
184
|
|
|
88
185
|
## Page format
|
|
89
186
|
|
package/src/config.ts
CHANGED
|
@@ -40,6 +40,7 @@ export interface JevWikiConfig {
|
|
|
40
40
|
autoAccept: number;
|
|
41
41
|
minSupport: number;
|
|
42
42
|
minDerivable: number;
|
|
43
|
+
framingImportance: number;
|
|
43
44
|
minNovelty: number;
|
|
44
45
|
minImportance: number;
|
|
45
46
|
};
|
|
@@ -65,7 +66,7 @@ export const DEFAULT_CONFIG: JevWikiConfig = {
|
|
|
65
66
|
routing: { shardSize: 250, minFit: 0.6, newPageConfidence: 0.7 },
|
|
66
67
|
review: { mode: "agent", escalateCriticality: 0.85, maxPerSession: 10 },
|
|
67
68
|
sync: { onSessionStart: "check", onCommit: false, backstopLintDays: 14 },
|
|
68
|
-
thresholds: { autoAccept: 0.8, minSupport: 0.7, minDerivable: 0.5, minNovelty: 0.6, minImportance: 1 },
|
|
69
|
+
thresholds: { autoAccept: 0.8, minSupport: 0.7, minDerivable: 0.5, framingImportance: 0.6, minNovelty: 0.6, minImportance: 1 },
|
|
69
70
|
weights: { grounded: 0.45, importance: 0.25, nonDerivable: 0.2, authority: 0.1 },
|
|
70
71
|
toc: { maxTokens: 3000 },
|
|
71
72
|
lint: { orphanMinAgeDays: 7, duplicateSimilarity: 0.72 },
|
package/src/extension.ts
CHANGED
|
@@ -23,6 +23,7 @@ import { applyReinforcement, applySupersession, bestCandidatePage } from "./prov
|
|
|
23
23
|
import { redact } from "./redact.ts";
|
|
24
24
|
import { appendSessionLog, promoteRecurring } from "./sessionlog.ts";
|
|
25
25
|
import { renderStructure, scanStructure } from "./structure.ts";
|
|
26
|
+
import { buildTriageReport, renderTriage } from "./triage.ts";
|
|
26
27
|
import { extractInsights, sessionTextFromEntries } from "./pipeline/capture.ts";
|
|
27
28
|
import {
|
|
28
29
|
applyReviewResolution,
|
|
@@ -966,8 +967,9 @@ export default function (pi: ExtensionAPI) {
|
|
|
966
967
|
promptSnippet: "Capture durable project insights from this session into the wiki",
|
|
967
968
|
promptGuidelines: [
|
|
968
969
|
"Use wiki_insights at the end of substantive work to capture durable, non-derivable knowledge (decisions, invariants, architecture, gotchas) with evidence pointers.",
|
|
970
|
+
"Before submitting, apply the pre-submission checklist in the llm-wiki skill (How claims are judged): the evidence must state the claim; attach the introducing commit for decisions and the user's own words for policy.",
|
|
969
971
|
"Do not capture transient task state, code snippets, or anything derivable by reading the repo.",
|
|
970
|
-
"After wiki_insights, write or merge the
|
|
972
|
+
"After wiki_insights, write or merge only the accepted pages, then call wiki_finalize. Use wiki_triage to diagnose rejected claims instead of writing them by hand.",
|
|
971
973
|
],
|
|
972
974
|
parameters: Type.Object({
|
|
973
975
|
insights: Type.Array(
|
|
@@ -1446,6 +1448,28 @@ export default function (pi: ExtensionAPI) {
|
|
|
1446
1448
|
},
|
|
1447
1449
|
});
|
|
1448
1450
|
|
|
1451
|
+
pi.registerTool({
|
|
1452
|
+
name: "wiki_triage",
|
|
1453
|
+
label: "Triage Rejected Insights",
|
|
1454
|
+
description:
|
|
1455
|
+
"Explain rejected wiki claims: why each was rejected, its Jev scores, whether the problem is evidence or policy, how to fix it, and whether the derivability threshold is calibrated for this project. No model calls.",
|
|
1456
|
+
promptSnippet: "Triage rejected insights and how to fix them",
|
|
1457
|
+
promptGuidelines: [
|
|
1458
|
+
"Use wiki_triage when an agent or user disagrees with rejected insights, and before re-submitting a rejection.",
|
|
1459
|
+
"After wiki_triage, re-submit fixable claims with wiki_insights using commit messages or quotes as evidence.",
|
|
1460
|
+
],
|
|
1461
|
+
parameters: Type.Object({
|
|
1462
|
+
limit: Type.Optional(Type.Number({ description: "Max rejections to list (default 20)" })),
|
|
1463
|
+
sinceDays: Type.Optional(Type.Number({ description: "Only consider rejections from the last N days" })),
|
|
1464
|
+
}),
|
|
1465
|
+
async execute(_id, params, _signal, _onUpdate, ctx) {
|
|
1466
|
+
const { layout } = runtimeFor(ctx);
|
|
1467
|
+
const ledger = await readLedger(layout);
|
|
1468
|
+
const report = buildTriageReport(ledger, { limit: params.limit ?? 20, sinceDays: params.sinceDays });
|
|
1469
|
+
return { content: [{ type: "text", text: renderTriage(report) }], details: report };
|
|
1470
|
+
},
|
|
1471
|
+
});
|
|
1472
|
+
|
|
1449
1473
|
// Commands -----------------------------------------------------------------
|
|
1450
1474
|
|
|
1451
1475
|
pi.registerCommand("wiki:status", {
|
|
@@ -371,6 +371,20 @@ export function decideClaim(verdicts: ClaimVerdicts, config: ResolvedConfig): Cl
|
|
|
371
371
|
return { action: "reject_sensitive", score: 0, reasons: ["contains sensitive content"] };
|
|
372
372
|
}
|
|
373
373
|
if (verdicts.derivable >= thresholds.minDerivable) {
|
|
374
|
+
const framingKinds = ["architecture", "invariant", "decision"];
|
|
375
|
+
const framing =
|
|
376
|
+
framingKinds.includes(verdicts.kind) &&
|
|
377
|
+
verdicts.importanceNorm >= thresholds.framingImportance &&
|
|
378
|
+
verdicts.grounded >= 0.5;
|
|
379
|
+
if (framing) {
|
|
380
|
+
return {
|
|
381
|
+
action: "review",
|
|
382
|
+
score: 0,
|
|
383
|
+
reasons: [
|
|
384
|
+
`derivable from code (${verdicts.derivable.toFixed(2)} ≥ ${thresholds.minDerivable}) but high-importance ${verdicts.kind} framing — queued for confirmation instead of dropped`,
|
|
385
|
+
],
|
|
386
|
+
};
|
|
387
|
+
}
|
|
374
388
|
return {
|
|
375
389
|
action: "reject_derivable",
|
|
376
390
|
score: 0,
|
package/src/triage.ts
ADDED
|
@@ -0,0 +1,190 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Triage for rejected claims.
|
|
3
|
+
*
|
|
4
|
+
* The decision ledger already holds every verdict. This module turns it into an
|
|
5
|
+
* actionable report: what was rejected, why, whether the rejection is fixable
|
|
6
|
+
* (re-submit with better evidence) or a policy call (high-importance framing),
|
|
7
|
+
* and whether the derivability threshold is calibrated for this project.
|
|
8
|
+
*/
|
|
9
|
+
import type { LedgerEntry } from "./ledger.ts";
|
|
10
|
+
|
|
11
|
+
export interface RejectionEntry {
|
|
12
|
+
ts: string;
|
|
13
|
+
claim: string;
|
|
14
|
+
action: string;
|
|
15
|
+
reason: string;
|
|
16
|
+
derivable?: number;
|
|
17
|
+
grounded?: number;
|
|
18
|
+
importance?: number;
|
|
19
|
+
kind?: string;
|
|
20
|
+
trustTier?: string;
|
|
21
|
+
}
|
|
22
|
+
|
|
23
|
+
export interface ScoreStats {
|
|
24
|
+
count: number;
|
|
25
|
+
min: number;
|
|
26
|
+
median: number;
|
|
27
|
+
max: number;
|
|
28
|
+
}
|
|
29
|
+
|
|
30
|
+
export interface TriageReport {
|
|
31
|
+
rejections: RejectionEntry[];
|
|
32
|
+
counts: Record<string, number>;
|
|
33
|
+
separated: {
|
|
34
|
+
acceptedDerivable: ScoreStats | null;
|
|
35
|
+
rejectedDerivable: ScoreStats | null;
|
|
36
|
+
rejectedGrounded: ScoreStats | null;
|
|
37
|
+
};
|
|
38
|
+
}
|
|
39
|
+
|
|
40
|
+
const ADJUDICATE_OPS = new Set(["insight.adjudicate", "ingest.adjudicate"]);
|
|
41
|
+
const ACCEPTED = new Set(["file", "reinforce", "file_user_stated"]);
|
|
42
|
+
|
|
43
|
+
function asRecord(value: unknown): Record<string, unknown> {
|
|
44
|
+
return value && typeof value === "object" ? (value as Record<string, unknown>) : {};
|
|
45
|
+
}
|
|
46
|
+
|
|
47
|
+
function asNumber(value: unknown): number | undefined {
|
|
48
|
+
return typeof value === "number" && Number.isFinite(value) ? value : undefined;
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
function stats(values: number[]): ScoreStats | null {
|
|
52
|
+
if (values.length === 0) return null;
|
|
53
|
+
const sorted = [...values].sort((a, b) => a - b);
|
|
54
|
+
return {
|
|
55
|
+
count: sorted.length,
|
|
56
|
+
min: Number(sorted[0].toFixed(2)),
|
|
57
|
+
median: Number(sorted[Math.floor(sorted.length / 2)].toFixed(2)),
|
|
58
|
+
max: Number(sorted[sorted.length - 1].toFixed(2)),
|
|
59
|
+
};
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
export function buildTriageReport(entries: LedgerEntry[], options?: { limit?: number; sinceDays?: number }): TriageReport {
|
|
63
|
+
const cutoff = options?.sinceDays ? Date.now() - options.sinceDays * 86_400_000 : undefined;
|
|
64
|
+
const rows = entries.filter((entry) => {
|
|
65
|
+
if (!ADJUDICATE_OPS.has(String(entry.op))) return false;
|
|
66
|
+
if (cutoff !== undefined) {
|
|
67
|
+
const ts = Date.parse(String(entry.ts ?? ""));
|
|
68
|
+
if (!Number.isFinite(ts) || ts < cutoff) return false;
|
|
69
|
+
}
|
|
70
|
+
return true;
|
|
71
|
+
});
|
|
72
|
+
|
|
73
|
+
const counts: Record<string, number> = {};
|
|
74
|
+
const rejections: RejectionEntry[] = [];
|
|
75
|
+
const acceptedDerivable: number[] = [];
|
|
76
|
+
const rejectedDerivable: number[] = [];
|
|
77
|
+
const rejectedGrounded: number[] = [];
|
|
78
|
+
|
|
79
|
+
for (const row of rows) {
|
|
80
|
+
const action = String(row.action ?? "unknown");
|
|
81
|
+
counts[action] = (counts[action] ?? 0) + 1;
|
|
82
|
+
const verdict = asRecord(row.verdict);
|
|
83
|
+
const derivable = asNumber(verdict.derivable);
|
|
84
|
+
const grounded = asNumber(verdict.grounded);
|
|
85
|
+
if (ACCEPTED.has(action) && derivable !== undefined) acceptedDerivable.push(derivable);
|
|
86
|
+
if (action === "reject_derivable" && derivable !== undefined) rejectedDerivable.push(derivable);
|
|
87
|
+
if (action === "reject_unsupported" && grounded !== undefined) rejectedGrounded.push(grounded);
|
|
88
|
+
if (!action.startsWith("reject")) continue;
|
|
89
|
+
rejections.push({
|
|
90
|
+
ts: String(row.ts ?? ""),
|
|
91
|
+
claim: String(row.subject ?? "(unknown claim)"),
|
|
92
|
+
action,
|
|
93
|
+
reason: String(row.reason ?? verdict.reason ?? ""),
|
|
94
|
+
derivable,
|
|
95
|
+
grounded,
|
|
96
|
+
importance: asNumber(verdict.importanceNorm),
|
|
97
|
+
kind: typeof verdict.kind === "string" ? verdict.kind : undefined,
|
|
98
|
+
trustTier: typeof verdict.trustTier === "string" ? verdict.trustTier : undefined,
|
|
99
|
+
});
|
|
100
|
+
}
|
|
101
|
+
|
|
102
|
+
rejections.sort((a, b) => b.ts.localeCompare(a.ts));
|
|
103
|
+
const limited = options?.limit ? rejections.slice(0, options.limit) : rejections;
|
|
104
|
+
|
|
105
|
+
return {
|
|
106
|
+
rejections: limited,
|
|
107
|
+
counts,
|
|
108
|
+
separated: {
|
|
109
|
+
acceptedDerivable: stats(acceptedDerivable),
|
|
110
|
+
rejectedDerivable: stats(rejectedDerivable),
|
|
111
|
+
rejectedGrounded: stats(rejectedGrounded),
|
|
112
|
+
},
|
|
113
|
+
};
|
|
114
|
+
}
|
|
115
|
+
|
|
116
|
+
/** Concrete next step for a rejected claim. */
|
|
117
|
+
export function remedyFor(entry: RejectionEntry): string {
|
|
118
|
+
switch (entry.action) {
|
|
119
|
+
case "reject_derivable": {
|
|
120
|
+
const framing = ["architecture", "invariant", "decision"].includes(entry.kind ?? "") && (entry.importance ?? 0) >= 0.6;
|
|
121
|
+
if (framing) {
|
|
122
|
+
return "High-importance framing: re-submit as-is (kind architecture/invariant/decision) and it will be queued for confirmation instead of dropped.";
|
|
123
|
+
}
|
|
124
|
+
return "Add rationale the code cannot show: a commit message (evidence kind=commit), a source quote, or state the reasoning behind it as kind=decision.";
|
|
125
|
+
}
|
|
126
|
+
case "reject_unsupported":
|
|
127
|
+
return "Ground it: attach a verbatim quote (kind=source), the commit message that introduced it (kind=commit), a file excerpt that states it, or the user's own words (kind=user). Referencing a file without a supporting passage is not enough.";
|
|
128
|
+
case "reject_duplicate":
|
|
129
|
+
return "Already covered: find the page, then reinforce it (wiki_insights with the page's claim) or extend it rather than filing a new claim.";
|
|
130
|
+
case "reject_sensitive":
|
|
131
|
+
return "Contains secrets or PII; redact before re-submitting. Sensitive claims are never written to the wiki.";
|
|
132
|
+
default:
|
|
133
|
+
return "Review the reason above and re-submit with stronger evidence if the claim matters.";
|
|
134
|
+
}
|
|
135
|
+
}
|
|
136
|
+
|
|
137
|
+
export function renderTriage(report: TriageReport): string {
|
|
138
|
+
const lines: string[] = ["# Rejected-claim triage", ""];
|
|
139
|
+
const total = Object.values(report.counts).reduce((sum, count) => sum + count, 0);
|
|
140
|
+
lines.push(
|
|
141
|
+
`Adjudicated: ${total} · accepted ${["file", "reinforce", "file_user_stated"].reduce((sum, action) => sum + (report.counts[action] ?? 0), 0)} · rejected ${Object.entries(report.counts)
|
|
142
|
+
.filter(([action]) => action.startsWith("reject"))
|
|
143
|
+
.reduce((sum, [, count]) => sum + count, 0)} · queued ${report.counts.review ?? 0}`,
|
|
144
|
+
"",
|
|
145
|
+
`Breakdown: ${Object.entries(report.counts)
|
|
146
|
+
.map(([action, count]) => `${action} ${count}`)
|
|
147
|
+
.join(" · ")}`,
|
|
148
|
+
"",
|
|
149
|
+
);
|
|
150
|
+
|
|
151
|
+
const { acceptedDerivable, rejectedDerivable, rejectedGrounded } = report.separated;
|
|
152
|
+
if (acceptedDerivable && rejectedDerivable) {
|
|
153
|
+
const overlap = acceptedDerivable.max >= rejectedDerivable.min;
|
|
154
|
+
lines.push(
|
|
155
|
+
`## Threshold separation (project data)`,
|
|
156
|
+
`- accepted claims derivable: ${acceptedDerivable.min}–${acceptedDerivable.max} (median ${acceptedDerivable.median}, n=${acceptedDerivable.count})`,
|
|
157
|
+
`- reject_derivable derivable: ${rejectedDerivable.min}–${rejectedDerivable.max} (median ${rejectedDerivable.median}, n=${rejectedDerivable.count})`,
|
|
158
|
+
overlap
|
|
159
|
+
? `- ⚠️ ranges overlap: the ${"minDerivable"} threshold may be miscalibrated for this project — consider raising thresholds.minDerivable or filing the borderline claims as framing.`
|
|
160
|
+
: `- clean separation: no accepted claim scored at or above the rejected range.`,
|
|
161
|
+
"",
|
|
162
|
+
);
|
|
163
|
+
}
|
|
164
|
+
if (rejectedGrounded) {
|
|
165
|
+
lines.push(
|
|
166
|
+
`- reject_unsupported grounded: ${rejectedGrounded.min}–${rejectedGrounded.max} (median ${rejectedGrounded.median}, n=${rejectedGrounded.count}) — these are evidence problems, not policy problems.`,
|
|
167
|
+
"",
|
|
168
|
+
);
|
|
169
|
+
}
|
|
170
|
+
|
|
171
|
+
if (report.rejections.length === 0) {
|
|
172
|
+
lines.push("No rejected claims in the selected window.");
|
|
173
|
+
return lines.join("\n");
|
|
174
|
+
}
|
|
175
|
+
|
|
176
|
+
lines.push("## Rejections (newest first)", "", "| When | Action | Claim | Scores | Fix |", "|------|--------|-------|--------|-----|");
|
|
177
|
+
for (const entry of report.rejections) {
|
|
178
|
+
const when = entry.ts.slice(0, 16).replace("T", " ");
|
|
179
|
+
const scores = [
|
|
180
|
+
entry.grounded !== undefined ? `g ${entry.grounded.toFixed(2)}` : "",
|
|
181
|
+
entry.derivable !== undefined ? `d ${entry.derivable.toFixed(2)}` : "",
|
|
182
|
+
entry.importance !== undefined ? `i ${entry.importance.toFixed(2)}` : "",
|
|
183
|
+
]
|
|
184
|
+
.filter(Boolean)
|
|
185
|
+
.join(" · ");
|
|
186
|
+
const claim = entry.claim.replace(/\|/g, "\\|").slice(0, 90);
|
|
187
|
+
lines.push(`| ${when} | ${entry.action} | ${claim} | ${scores || "—"} | ${remedyFor(entry)} |`);
|
|
188
|
+
}
|
|
189
|
+
return lines.join("\n");
|
|
190
|
+
}
|