@tangle-network/agent-knowledge 1.8.0 → 1.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,428 @@
1
+ # A verifier agent mostly deduplicates: a controlled A/B on two-agent web research, and what its cost buys
2
+
3
+ *Tangle Network · `agent-knowledge`*
4
+
5
+ ## Abstract
6
+
7
+ We test whether adding a second "verifier" agent to a web-research loop produces a
8
+ cleaner knowledge base than a single agent doing the same work, with compute held
9
+ fixed. A *worker* agent searches the web and proposes sources for the knowledge
10
+ base's open gaps; a *driver* agent vets each proposed source before it commits,
11
+ fills gaps the worker missed, and decides when the base is complete. Over 9
12
+ machine-learning topics at equal compute, the two-agent loop admitted **2.3–2.7
13
+ fewer sources per topic at identical coverage** — 95% bootstrap intervals
14
+ [1.78, 2.89] and [2.22, 3.00] across two independent runs, both above zero. The
15
+ effect is real and reproduces. But the mechanism is not the one we set out to
16
+ test: reading the rejection logs, most of the gain is **de-duplication** — the
17
+ same paper fetched from arXiv, OpenReview, and the NeurIPS proceedings — not the
18
+ relevance filtering we expected. Pricing the verifier's calls (we added per-arm
19
+ router-usage instrumentation) shows the cleanliness costs roughly **5× the
20
+ dollars, 9× the tokens, and 3× the latency** of the single agent — and that the
21
+ original "equal passes" framing hid this, because it charged the verify step as
22
+ one pass while it is actually one LLM call per proposed source. Since the win is
23
+ de-dup-dominated, a deterministic content hash recovers most of the cleanliness at
24
+ ~none of the premium. We then asked the sharper question — is there an error band
25
+ where an LLM verifier *does* earn its dollar? — and found two, on opposite sides of
26
+ the ledger. **Misattributed citations** (an on-topic, unique, real source whose
27
+ cited claim never appears in the page) are caught by a $0 deterministic
28
+ text-presence check that the LLM relevance judge misses 1 in 5 times, because the
29
+ judge structurally never sees the claim. And we built the deployable shape the
30
+ cost result implied — an **adaptive driver** that runs free dedup, then free
31
+ heuristic triage, and escalates to the LLM only on the ambiguous tail: it cuts
32
+ LLM verifier calls 76% and dollars 74%, recovering the de-dup half of the
33
+ verifier's cleanliness while honestly giving up the relevance-judgment half on a
34
+ source pool dominated by authoritative hosts. The verifier earns its dollar on
35
+ misattribution, not on de-duplication; the right production loop spends it only
36
+ where the cheap signals can't decide.
37
+
38
+ ## 1. Setup
39
+
40
+ A research agent building a knowledge base accumulates sources. A single agent both
41
+ finds sources and, implicitly, decides which to keep — it grades its own work. The
42
+ hypothesis is the usual one for verification: separating the producer (find
43
+ sources) from the checker (keep the good ones) should yield a cleaner result, for
44
+ the same reason a second pair of eyes catches typos the author misses.
45
+
46
+ The trap in any "more agents help" claim is compute. Two agents that simply do more
47
+ work will of course produce more — that is a bigger budget, not a finding. So the
48
+ comparison must hold total compute fixed and ask whether the *topology* — splitting
49
+ find from check — beats spending the same compute on a single agent that just finds
50
+ more. And once topology shows an effect, the second question is what it costs: a
51
+ cleaner base bought at 5× the inference is a different product decision than one
52
+ bought for free.
53
+
54
+ ## 2. Method
55
+
56
+ ### 2.1 The loop
57
+
58
+ The loop has two roles (`src/two-agent-research-loop.ts`, `runTwoAgentResearchLoop`):
59
+
60
+ - **Worker** — primary research. Each round it reads the open gaps and proposes
61
+ sources to close them (`ResearchWorker: (ctx: { gaps, steer }) => proposals`).
62
+ - **Driver** — does three things (`ResearchDriver`): `verifySource` vets each
63
+ proposed source before it commits (dedup against the base, then reject sources
64
+ that aren't relevant); `research` runs the driver's own gap-fill pass over gaps
65
+ the worker missed; `foldGaps` turns the still-open gaps into a `steer` string for
66
+ the worker's next round.
67
+
68
+ A round is therefore: worker proposes → driver verifies each proposal (rejections
69
+ never reach the base) → driver gap-fills → the readiness gate checks the base →
70
+ remaining gaps are folded into the next worker prompt. The loop stops when the gate
71
+ reports no blocking gaps left.
72
+
73
+ Note what the driver→worker hand-off is and isn't: the driver *steers* the worker
74
+ by handing it the remaining readiness gaps (`foldGaps`), which is a deterministic
75
+ formatting of unmet requirements — not an LLM authoring a fresh instruction. The
76
+ driver's LLM work is in `verifySource` (one call per proposed source) and its own
77
+ `research` pass. This matters for §4.2: the verify step is N calls, not one, and
78
+ the cost framing turns on that.
79
+
80
+ The readiness gate is `scoreKnowledgeReadiness` (from `agent-eval`). It scores
81
+ *pages* (curated `knowledge/*.md`), not raw sources, and only `importance:
82
+ 'blocking'` requirements gate. Coverage below is the fraction of a topic's blocking
83
+ requirements met.
84
+
85
+ ### 2.2 What the agents are — an honest note
86
+
87
+ The agents in the live run are **not** `AgentProfile`s on a coding harness. The
88
+ worker is a hand-wired pipeline (`src/web-research-worker.ts`,
89
+ `createWebResearchWorker`): glm-5.2 turns the gaps into search queries → a real web
90
+ search over the Tangle router (`POST /v1/search`) → each hit is fetched with the
91
+ repo's `politeFetch` and reduced to text with `htmlToText` → citing pages are
92
+ proposed. It talks to the router directly through `createTangleRouterClient` — no
93
+ claude-code / opencode / sandbox harness, and no dynamic harness selection. The
94
+ driver (`createVerifyingResearchDriver`) is one glm-5.2 chat call per source.
95
+
96
+ The repo *does* ship a real `AgentProfile` for research (`researcherProfile`), and
97
+ the **offline** control arm uses it with a stub harness — but the live arm bypasses
98
+ it for the direct pipeline. This is a deliberate shortcut (no harness to stand up,
99
+ ~$0.20 to run) and also the loop's main simplification debt; see §7.
100
+
101
+ ### 2.3 Equal compute
102
+
103
+ Compute is counted in agent passes. A two-agent round = 1 worker pass + 1
104
+ verify pass = 2 passes; a single-agent iteration = 1 pass. Both arms gate on the
105
+ *same* readiness criterion and stop as soon as it is met, so neither is starved.
106
+ We budget-match by passes, not rounds — the single-agent loop gets more rounds to
107
+ spend the compute the two-agent loop spends on verification. The harness asserts,
108
+ per topic, that the two-agent loop spent no more passes than the single-agent loop
109
+ and that both stayed under the ceiling; if that ever fails the comparison has
110
+ drifted to unequal compute and the result is void.
111
+
112
+ The pass-accounting has a known soft spot, which §4.2 exposes: a "verify pass" is
113
+ not one LLM call, it is one `verifySource` call per *proposed* source that round.
114
+ Charging it as a single pass keeps the topology comparison fair on agent passes but
115
+ understates the verifier's dollar cost. We added explicit per-arm cost
116
+ instrumentation to measure that directly.
117
+
118
+ ### 2.4 Cost instrumentation
119
+
120
+ The router client now records usage per call (`RouterClient.usage()`,
121
+ `src/web-research-worker.ts`): cumulative chat-completion count, prompt/completion
122
+ tokens, glm-5.2 priced cost, and wall latency. Each A/B arm reads the accumulator
123
+ before and after its run and diffs, so every reported dollar and token figure is a
124
+ measured per-arm delta, not an estimate.
125
+
126
+ ### 2.5 Topics and readiness
127
+
128
+ 9 topics, each with two blocking requirements (the defining mechanism, and reported
129
+ results / trade-offs). Seven are "narrow-scope-inside-a-broad-space" (e.g.
130
+ *self-speculative decoding* inside *speculative decoding*), where we expected the
131
+ broad space to leak in; two are clean controls (*the transformer architecture*,
132
+ *gradient descent*).
133
+
134
+ ## 3. Result 1 — cleanliness: the verifier admits fewer sources at equal coverage
135
+
136
+ The cleanliness signal is the **admitted-source count**: on live data there is no
137
+ oracle, so "fewer sources admitted at equal coverage" is the measurable proxy for
138
+ "cleaner." Δ = single-agent admitted − two-agent admitted, per topic.
139
+
140
+ | Topic | band | Run 1 Δ | Run 2 Δ |
141
+ |---|---|---|---|
142
+ | self-speculative decoding | narrow | 3 | 3 |
143
+ | grouped-query attention | narrow | 3 | 3 |
144
+ | rotary position embeddings | narrow | 2 | 3 |
145
+ | KV-cache quantization | narrow | 1 | 1 |
146
+ | LoRA | narrow | 1 | 3 |
147
+ | ring attention | narrow | 2 | 3 |
148
+ | constitutional AI | narrow | 3 | 3 |
149
+ | the transformer architecture | clean | 3 | 3 |
150
+ | gradient descent | clean | 3 | 2 |
151
+ | **mean Δ** | | **2.33** | **2.67** |
152
+ | **95% CI** (paired bootstrap) | | **[1.78, 2.89]** | **[2.22, 3.00]** |
153
+
154
+ Coverage was **1.00 on every topic, both arms, both runs** — the verifier never
155
+ cost completeness. Both bootstrap intervals (`pairedBootstrap`, from `agent-eval`)
156
+ are above zero. The effect reproduces; its exact magnitude varies run-to-run with
157
+ what the web returns (one topic swung Δ = 0→1→3 across separate runs during
158
+ development).
159
+
160
+ A third, cost-instrumented run (the one priced in §4.2) was noisier: mean Δ **+1.56,
161
+ 95% CI [0.33, 2.67]**, with two topics where the two-agent loop admitted *more* than
162
+ the single agent (KV-cache quantization −1, gradient descent −1). The interval still
163
+ clears zero, but it is the lower-bound run — a reminder that the magnitude is
164
+ web-variance-bound, while the sign is stable.
165
+
166
+ ## 4. Result 2 — what the verifier does, and what it costs
167
+
168
+ ### 4.1 Mostly de-duplication
169
+
170
+ We classified each rejection by the verifier's own stated reason:
171
+
172
+ | rejection reason | narrow (7) | clean (2) |
173
+ |---|---|---|
174
+ | near-duplicate (same paper, different host) | 6 | 4 |
175
+ | off-scope (broad space leaked in) | 3 | 2 |
176
+ | junk page (aggregator / marketing / explainer) | 3 | 0 |
177
+
178
+ The dominant mechanism is **de-duplication** — canonical papers mirrored across
179
+ arXiv, OpenReview, and the NeurIPS proceedings — and it fires regardless of band.
180
+ The off-scope rejection we set out to measure is real (on *self-speculative
181
+ decoding* the verifier correctly dropped three general *speculative decoding* papers
182
+ that use a *separate* draft model) but it is the minority, and it does not
183
+ concentrate on the narrow topics as hypothesized: narrow mean Δ = 2.14 vs clean
184
+ 3.00. The strong form of our hypothesis — narrow-in-broad pays more — is **refuted
185
+ in magnitude, confirmed only in mechanism**.
186
+
187
+ The practical reading: most of the win is "you fetched the same PDF three times,"
188
+ which a content hash catches for free. The LLM's distinctive contribution is the
189
+ page that *looks* on-topic but isn't — the self-speculative-vs-separate-draft
190
+ distinction a string match would miss.
191
+
192
+ ### 4.2 The inference premium (and what "equal passes" hid)
193
+
194
+ `docs/results/cost-quality.md`. The original A/B reported only admitted-sources at
195
+ "equal passes," which charged the verify step as one pass while it is actually N
196
+ `verifySource` LLM calls. Pricing the calls per arm (B ≤ 4 passes/arm, glm-5.2):
197
+
198
+ | per topic (mean) | two-agent | single-agent | ratio |
199
+ |---|---|---|---|
200
+ | LLM chat calls | 5.4 | 1.0 | ~5.4× |
201
+ | tokens (in+out) | ~4,900 | ~530 | ~9× |
202
+ | cost (USD) | ~$0.0072 | ~$0.0013 | ~5.5× |
203
+ | latency (wall) | ~37 s | ~11 s | ~3.4× |
204
+ | cleanliness Δ (single − two admitted) | — | — | +1.56, 95% CI [0.33, 2.67] |
205
+
206
+ The verifier buys ~1.5–2.7 fewer junk sources for roughly **5× the dollars, 9× the
207
+ tokens, and 3× the latency**. Since the cleanliness gain is de-dup-dominated
208
+ (§4.1), the honest production move is a deterministic content-hash / canonical-URL
209
+ dedup, which captures most of the cleanliness at ~none of this premium, reserving
210
+ an LLM check only for the off-scope tail. This is the cost half the "equal passes"
211
+ framing left out — and the rest of the paper is what we built once we saw it.
212
+
213
+ ## 5. Result 3 — the two bands where an LLM verifier does, and doesn't, earn its dollar
214
+
215
+ If de-dup is free and dominates the win, when is the LLM verifier worth its 5×? We
216
+ found two bands, and they cut in opposite directions.
217
+
218
+ ### 5.1 Misattributed citations — the cheap check beats the expensive judge
219
+
220
+ `docs/results/claim-grounding.md`. A source can be on-topic, unique, and real, yet
221
+ the cited *claim* never appears in the page — the LLM wrote a plausible sentence and
222
+ hung a real URL off it. De-dup passes it (unique). A relevance judge passes it (the
223
+ page is on-topic). Only checking the claim against the fetched text catches it — and
224
+ that check is **deterministic text presence, $0 inference**.
225
+
226
+ Each proposed source now carries the claim it is cited for (`withCitedClaim` →
227
+ `metadata.citedClaim`). The claim-grounding verifier (`createClaimGroundingVerifier`,
228
+ `src/claim-grounding.ts`) runs `groundClaimInText(claim, pageText)` over the
229
+ `htmlToText` output of the page the worker actually fetched — verbatim, normalized
230
+ (punctuation/whitespace-insensitive), or a ≥70% content-word overlap close
231
+ paraphrase. A claim that isn't present is rejected as misattributed. The oracle is
232
+ text presence, not a model call, so it composes with the LLM relevance verifier or
233
+ runs alone at zero cost.
234
+
235
+ Live A/B (glm-5.2, real web fetch, one planted misattribution per topic — a real
236
+ fetched page plus a deliberately-wrong claim — over three verifier arms on the same
237
+ proposals):
238
+
239
+ | n=5 topics | misattributions caught | marginal $ | per-$ caught |
240
+ |---|---|---|---|
241
+ | no-verifier | 0 / 5 | $0.0000 | — |
242
+ | relevance (LLM judge) | 4 / 5 | $0.0157 | 254 |
243
+ | claim-grounding (text) | **5 / 5** | **$0.0000** | ∞ |
244
+
245
+ The relevance judge catches one only by accident — when the fabricated claim also
246
+ makes the page read off-topic (a "12-billion-parameter draft transformer" claim on a
247
+ rotary-embeddings page). When the fabrication stays on-topic (the KV-cache case), the
248
+ judge waves it through, because the relevance verifier only ever sees the page text,
249
+ never the cited claim — it is **structurally blind** to misattribution. On this band
250
+ the verifier-per-dollar comparison inverts §4.2: the cheap, deterministic check
251
+ catches strictly more (5/5 vs 4/5) at strictly less ($0 vs $0.0157). The offline
252
+ floor confirms the wiring: on a controlled 4-source pool (2 grounded, 2
253
+ misattributed), claim-grounding admits **0/2** misattributions and keeps **2/2**
254
+ grounded, while relevance and no-verifier both admit **2/2**.
255
+
256
+ ### 5.2 Adaptive topology — pay the LLM only on the ambiguous tail
257
+
258
+ `docs/results/adaptive.md`. The deployable shape §4.2 implied: do the free
259
+ deterministic work first, reserve the LLM for what the cheap signals can't decide.
260
+ `createAdaptiveResearchDriver` (`src/adaptive-driver.ts`) is that driver. Per
261
+ candidate source it runs three stages, cheapest first, stopping at the first that
262
+ decides:
263
+
264
+ 1. **Dedup ($0).** Reject a source whose canonical URL (scheme / `www` / trailing
265
+ slash / tracking params stripped) or normalized-text content hash matches one
266
+ already accepted this round or in the KB.
267
+ 2. **Heuristic triage ($0).** Classify a unique survivor with host/title/length
268
+ signals only: an authoritative host (arxiv, `*.edu`, `*.gov`, official docs,
269
+ github, …) with a substantial body is **kept**; an obvious spam/listicle title
270
+ or a too-thin body is **dropped**; everything else is **ambiguous**.
271
+ 3. **LLM escalation ($).** Only ambiguous survivors reach the shipped LLM relevance
272
+ verifier — one call each.
273
+
274
+ Live frontier, n=5 topics, glm-5.2, same fetched proposals gated through all three
275
+ drivers (plus one planted tracking-decorated mirror of the first source, so the
276
+ dedup stage has a real duplicate to catch). Total spend $0.033:
277
+
278
+ | topic | fetched | single admit | full-LLM admit / calls / $ | adaptive admit / LLM calls / $ |
279
+ |---|---|---|---|---|
280
+ | self-speculative decoding | 3 | 3 | 1 / 3 / $0.0027 | 2 / **0** / **$0.0000** |
281
+ | rotary position embeddings | 3 | 3 | 1 / 3 / $0.0031 | 2 / **0** / **$0.0000** |
282
+ | grouped-query attention | 7 | 7 | 3 / 7 / $0.0072 | 6 / 3 / $0.0030 |
283
+ | KV-cache quantization | 5 | 5 | 3 / 5 / $0.0052 | 4 / **0** / **$0.0000** |
284
+ | LoRA fine-tuning | 7 | 7 | 4 / 7 / $0.0079 | 6 / 3 / $0.0037 |
285
+ | **total** | **25** | **25** | **12 / 25 / $0.0261** | **20 / 6 / $0.0068** |
286
+
287
+ Adaptive cuts LLM verifier calls **76%** (25 → 6) and dollars **74%** ($0.0261 →
288
+ $0.0068). On 3 of the 5 topics it spent **zero** LLM calls — every unique survivor
289
+ was on an authoritative host, so the $0 stages decided everything.
290
+
291
+ It is a frontier point, not a free lunch. Admitted counts (lower = cleaner): single
292
+ 25, **adaptive 20**, full-LLM 12. Adaptive removes the 5 real duplicates the $0 dedup
293
+ catches — exactly the de-dup-dominated win — but keeps the 8 sources full-LLM rejects
294
+ on relevance, because on this authoritative-host-heavy set the heuristic resolved
295
+ every non-duplicate survivor without ever asking the LLM, and the host prior is
296
+ coarser than the relevance judge. So adaptive **recovers the deterministic de-dup
297
+ half of full-LLM's cleanliness for ~26% of its dollars, and gives up the
298
+ relevance-judgment half**. The escalation count is the diagnostic: on the 3 topics
299
+ where it was zero, adaptive *is* a pure host/title/length rule and the LLM
300
+ contributes nothing by construction; on the 2 topics with unknown-host survivors
301
+ (grouped-query attention, LoRA) it escalated 3 calls each — the off-scope tail the
302
+ verifier is actually for.
303
+
304
+ ## 6. Discussion
305
+
306
+ The three results compose into one rule. The LLM verifier's headline cleanliness win
307
+ is real (§3) but **de-dup-dominated** (§4.1) and **expensive** (§4.2, ~5×/9×/3×), so
308
+ spending an LLM call on every source is the wrong default — a free content hash buys
309
+ most of it. The verifier earns its 5× exactly where the cheap signals are blind: on
310
+ **misattribution** (§5.1), where a $0 text-presence check beats the LLM judge
311
+ outright because the judge never sees the claim; and on the **off-scope tail** (§5.2),
312
+ where a page looks on-topic, is unique, and isn't fabricated, so only a relevance
313
+ judgment can settle it. The deployable loop therefore stratifies by cost: free dedup,
314
+ free claim-grounding, free heuristic triage, then an LLM call only on what survives —
315
+ which is what the adaptive driver ships.
316
+
317
+ Two cross-cutting lessons. First, **the accounting unit decides the verdict**:
318
+ charging the verify step as one pass made the topology look near-free; pricing it per
319
+ LLM call (§4.2) is what surfaced the 5× and motivated everything after it. Second,
320
+ **the same verifier inverts in value across bands** — on de-dup the LLM is expensive
321
+ for what a hash does; on misattribution a deterministic check is free for what the LLM
322
+ can't do; on the off-scope tail the LLM is the only thing that works. "Add a verifier"
323
+ is not a setting; it is a cost-stratified decision per error type.
324
+
325
+ ## 7. Limitations
326
+
327
+ - **The verifier is also the judge.** Admitted-count is a proxy; we have no
328
+ independent oracle for whether a dropped source was genuinely redundant. The
329
+ verifier's stated reasons hold up on inspection, but this is the load-bearing
330
+ caveat for §3–§4.
331
+ - **Deltas are conservative.** The single-agent loop stops on the same readiness
332
+ gate, capping its admits; with more iterations it would admit even more junk, so
333
+ the true gap is at least this large.
334
+ - **Small n.** n = 2 clean controls is too thin to compare bands; the misattribution
335
+ and adaptive frontiers are n = 5 each. The directions are asserted in the tests on
336
+ every run; the magnitudes are small-n and web-variance-bound (the §3 third run swung
337
+ to +1.56 from +2.3/+2.7).
338
+ - **Planted error bands.** The misattributions (§5.1) and the adaptive duplicate
339
+ (§5.2) are injected so the band is measurable. They model the real LLM
340
+ citation-fabrication and mirror-host failures but do not measure their base rate in
341
+ the wild — that needs a hand-checked corpus of model-written citations.
342
+ - **Adaptive's quality is host-prior-bound.** On an authoritative-host-heavy source
343
+ pool the heuristic resolves everything and the LLM's relevance judgment contributes
344
+ nothing; a richer worker (good sources on unknown hosts, junk on on-topic-looking
345
+ pages) would grow the ambiguous tail and converge adaptive toward full-LLM cost.
346
+ - **glm-5.2-specific.** A weaker or stronger judge would shift rejection rates and the
347
+ relevance miss-rate. The grounding oracle is also conservative: a real paraphrase
348
+ whose inflected words differ ("drafts" vs "draft") can fall below the 0.7 overlap and
349
+ be flagged misattributed; `minOverlap` tunes this.
350
+ - **High web variance.** One live run per topic per result; numbers move with what
351
+ search returns.
352
+
353
+ ## 8. A simpler loop — built, not deferred
354
+
355
+ The original write-up named two simplifications as future work. Both are now built and
356
+ measured; this is what changed.
357
+
358
+ 1. **Deterministic dedup before the LLM, LLM only on the tail — shipped.** The
359
+ adaptive driver (`src/adaptive-driver.ts`, §5.2) does exactly this: free
360
+ canonical-URL / content-hash dedup, free host/title/length triage, LLM relevance
361
+ only on the ambiguous survivors. Measured: **76% fewer LLM calls, 74% cheaper**,
362
+ recovering the de-dup half of the verifier's cleanliness. The remaining gap to
363
+ full-LLM is the relevance-judgment half, kept honest in §5.2 — adaptive is a
364
+ frontier point you choose by how much a kept-but-marginal source costs you, not a
365
+ strict improvement.
366
+ 2. **A free check the LLM judge can't replicate — shipped.** Claim-grounding
367
+ (`src/claim-grounding.ts`, §5.1) adds the one verification an LLM relevance judge is
368
+ structurally blind to: does the cited claim actually appear in the page? It catches
369
+ 5/5 planted misattributions at **$0**, vs the judge's 4/5 at ~$0.003/topic.
370
+
371
+ What is still **not** built remains the worker: the live worker is a ~500-line
372
+ hand-wired pipeline (query-gen, search, fetch, propose) against the router directly,
373
+ where the repo's own pattern is to *author* an `AgentProfile` (`researcherProfile`)
374
+ and run it on a harness with a web-search tool — reusable and harness-agnostic. The
375
+ direct pipeline is cheaper to run today (no harness, no creds beyond the router) but it
376
+ is the loop's main remaining piece of duplication, and the obvious next step if this
377
+ loop graduates from experiment to production.
378
+
379
+ ## 9. Reproduce
380
+
381
+ The loop, the worker, the verifier, the claim-grounding mode, the adaptive driver, the
382
+ cost instrumentation, and every A/B are all in this repository. Each live test gates a
383
+ cheap one-call glm-5.2 smoke before any multi-topic burn.
384
+
385
+ ```bash
386
+ git clone https://github.com/tangle-network/agent-knowledge
387
+ cd agent-knowledge && pnpm install
388
+
389
+ # offline A/B — deterministic, no credentials (a controlled lower bound that
390
+ # exercises the same harness against a planted source pool)
391
+ pnpm exec vitest run tests/loops/research-loop-equal-compute.test.ts
392
+
393
+ # offline claim-grounding + adaptive floors (no credentials)
394
+ pnpm exec vitest run tests/loops/claim-grounding-ab.test.ts -t "offline"
395
+ pnpm exec vitest run tests/loops/adaptive-ab.test.ts
396
+
397
+ # the live cleanliness sweep — real web search + a real glm-5.2 verifier, with
398
+ # per-arm cost reported (~$0.20 for 9 topics)
399
+ export TANGLE_API_KEY=<router key with glm-5.2 credits>
400
+ AGENT_KNOWLEDGE_LIVE=1 \
401
+ AGENT_KNOWLEDGE_LIVE_GOALS="self-speculative decoding|grouped-query attention|rotary position embeddings|KV-cache quantization|LoRA|ring attention|constitutional AI|the transformer architecture|gradient descent" \
402
+ pnpm exec vitest run tests/loops/research-loop-equal-compute.test.ts
403
+
404
+ # live misattribution band — three verifier arms over the same proposals
405
+ AGENT_KNOWLEDGE_LIVE=1 TANGLE_API_KEY=<…> \
406
+ CLAIM_GROUNDING_LIVE_GOALS='self-speculative decoding|rotary position embeddings|grouped-query attention|KV-cache quantization|LoRA' \
407
+ pnpm exec vitest run tests/loops/claim-grounding-ab.test.ts -t "three verifier arms"
408
+
409
+ # live adaptive frontier — single / full-LLM / adaptive on the same fetched proposals
410
+ AGENT_KNOWLEDGE_LIVE=1 TANGLE_API_KEY=<…> \
411
+ ADAPTIVE_LIVE_GOALS="self-speculative decoding|rotary position embeddings|grouped-query attention|KV-cache quantization|LoRA fine-tuning" \
412
+ pnpm exec vitest run tests/loops/adaptive-ab.test.ts -t "three-topology"
413
+ ```
414
+
415
+ `AGENT_KNOWLEDGE_LIVE_GOALS` (and the per-result `*_LIVE_GOALS`) take a `|`-separated
416
+ topic list; the live arms run the loops on each at equal compute and report the paired
417
+ bootstrap and per-arm cost.
418
+
419
+ **Source:** the loop — [`src/two-agent-research-loop.ts`](../src/two-agent-research-loop.ts);
420
+ the live worker + verifier + cost instrumentation — [`src/web-research-worker.ts`](../src/web-research-worker.ts);
421
+ the misattribution check — [`src/claim-grounding.ts`](../src/claim-grounding.ts);
422
+ the adaptive driver — [`src/adaptive-driver.ts`](../src/adaptive-driver.ts);
423
+ the A/B harnesses — [`tests/loops/`](../tests/loops/).
424
+ Per-result detail: [`docs/results/cost-quality.md`](results/cost-quality.md),
425
+ [`docs/results/claim-grounding.md`](results/claim-grounding.md),
426
+ [`docs/results/adaptive.md`](results/adaptive.md).
427
+ </content>
428
+ </invoke>
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@tangle-network/agent-knowledge",
3
- "version": "1.8.0",
3
+ "version": "1.10.0",
4
4
  "description": "Source-grounded, eval-gated knowledge growth primitives for agents.",
5
5
  "homepage": "https://github.com/tangle-network/agent-knowledge#readme",
6
6
  "repository": {