loki-mode 7.129.5 → 8.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +121 -9
- package/SKILL.md +66 -5
- package/VERSION +1 -1
- package/autonomy/app-runner.sh +37 -0
- package/autonomy/completion-council.sh +862 -28
- package/autonomy/council-v2.sh +39 -0
- package/autonomy/crash.sh +3 -2
- package/autonomy/grill.sh +42 -0
- package/autonomy/hooks/validate-bash.sh +301 -4
- package/autonomy/lib/cockpit-render.sh +8 -2
- package/autonomy/lib/cr-rematerialize.py +101 -4
- package/autonomy/lib/deadline.py +957 -0
- package/autonomy/lib/dependency-setup.sh +205 -0
- package/autonomy/lib/done-recognition.sh +45 -0
- package/autonomy/lib/efficiency_cost.py +13 -6
- package/autonomy/lib/no_mock_scan.py +793 -0
- package/autonomy/lib/prd-enrich.sh +42 -0
- package/autonomy/lib/proof-analytics-props.py +69 -0
- package/autonomy/lib/proof-generator.py +326 -92
- package/autonomy/lib/proof-template.html +15 -12
- package/autonomy/lib/proof-verify.py +169 -105
- package/autonomy/lib/requirements_contract.py +848 -0
- package/autonomy/lib/sdk-mode.sh +103 -0
- package/autonomy/lib/secret-scan.sh +139 -0
- package/autonomy/lib/spec-expand.sh +208 -0
- package/autonomy/lib/tree_digest.py +266 -0
- package/autonomy/lib/voter-agents.sh +58 -8
- package/autonomy/lib/workspace_diff.py +141 -0
- package/autonomy/loki +196 -17
- package/autonomy/playwright-verify.sh +501 -0
- package/autonomy/prd-checklist.sh +17 -8
- package/autonomy/provider-offer.sh +111 -0
- package/autonomy/run.sh +4587 -671
- package/autonomy/sandbox.sh +42 -0
- package/autonomy/spec-interrogation.sh +148 -5
- package/autonomy/spec.sh +118 -1
- package/autonomy/telemetry.sh +133 -7
- package/autonomy/verify.sh +107 -0
- package/bin/loki +110 -3
- package/completions/_loki +10 -0
- package/completions/loki.bash +1 -1
- package/dashboard/__init__.py +1 -1
- package/dashboard/audit.py +96 -7
- package/dashboard/build_supervisor.py +1619 -0
- package/dashboard/control.py +8 -0
- package/dashboard/prompt_optimizer.py +7 -0
- package/dashboard/server.py +577 -22
- package/dashboard/static/trust.html +1 -1
- package/docs/ARCHITECTURE-OVERVIEW.md +207 -0
- package/docs/AUTONOMI-ECOSYSTEM.md +107 -0
- package/docs/INSTALLATION.md +19 -2
- package/docs/MODEL-EQUIVALENCE-HARNESS-PLAN.md +70 -0
- package/docs/PLANS-INDEX.md +33 -0
- package/docs/PRIVACY.md +29 -1
- package/docs/SIGNED-RECEIPTS.md +102 -0
- package/docs/SONNET5-DEFAULT-PLAN.md +1 -1
- package/docs/V8-ACCEPTANCE-TRIAGE-2026-07-24.md +114 -0
- package/docs/V8-AGENT-SDK-PLAN.md +692 -0
- package/docs/V8-COMPLEXITY-AUDIT-2026-07-24.md +45 -0
- package/docs/V8-MAJOR-RELEASE-PLAN.md +137 -0
- package/docs/V8-OVERNIGHT-PLAN-2026-07-25.md +82 -0
- package/docs/V8-RUNTIME-TRUTH-2026-07-25.md +232 -0
- package/docs/V8-SDK-RESEARCH-RAW.md +129 -0
- package/docs/alternative-installations.md +4 -4
- package/loki-ts/dist/loki.js +674 -285
- package/loki-ts/package.json +2 -0
- package/mcp/__init__.py +1 -1
- package/package.json +7 -6
- package/plugins/loki-mode/.claude-plugin/plugin.json +1 -1
- package/providers/claude.sh +75 -0
- package/providers/codex.sh +85 -13
- package/references/sdk-mode.md +106 -0
- package/skills/model-selection.md +1 -1
- package/skills/providers.md +42 -0
- package/skills/quality-gates.md +22 -0
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# v8 complexity audit: delete-list candidates
|
|
2
|
+
|
|
3
|
+
Date: 2026-07-24. Read-only. **Zero deletions performed** (founder decision).
|
|
4
|
+
|
|
5
|
+
## Why this audit exists
|
|
6
|
+
|
|
7
|
+
The competitive research corpus (source 12) proposed deleting `voice.sh`,
|
|
8
|
+
cockpit image rendering, `web-app/`, Managed Agents, and the "41 static roles"
|
|
9
|
+
to reduce single-maintainer load. VALIDATION.md marks that source's "unused"
|
|
10
|
+
label as an **assumption it never verified**, and the same source was already
|
|
11
|
+
caught undercounting `autonomy/loki` by 10k lines. So each candidate needed
|
|
12
|
+
positive proof, not a confident assertion.
|
|
13
|
+
|
|
14
|
+
**Method:** positive proof of non-reference. An absent grep hit is not evidence
|
|
15
|
+
(memory: grep-absence-false-green). A component counts as live if anything
|
|
16
|
+
references it, dispatches it, or ships it.
|
|
17
|
+
|
|
18
|
+
## Findings
|
|
19
|
+
|
|
20
|
+
| Candidate | Verdict | Evidence |
|
|
21
|
+
|---|---|---|
|
|
22
|
+
| `autonomy/voice.sh` (501 lines) | **ORPHANED (only true positive)** | Referenced by NOTHING except itself: a repo-wide search across `autonomy/`, `loki-ts/`, `scripts/`, `tests/`, `mcp/`, `dashboard/` returns only the file. Meanwhile `cmd_voice()` (`autonomy/loki:22751`) is dispatched (`:17884`) but is a 4-line stub that prints "Voice mode is planned for a future release" and points at issue #85. So the COMMAND is live and honest; the 501-line SCRIPT behind it is dead weight shipped in the npm tarball (`package.json` includes `autonomy/`). |
|
|
23
|
+
| cockpit rendering | **LIVE** | Real module with 5 source files (`loki-ts/src/cockpit/{raster,cli,capability,svg,render}.ts`). Not an orphan. Source 12 cited `loki-ts/src/commands/cockpit.ts`, which does NOT exist: its path was wrong, its conclusion untested. |
|
|
24
|
+
| `web-app/` | **LIVE AND SHIPPED** | 12 explicit entries in `package.json` `files` (`web-app/dist/`, `server.py`, `auth.py`, `models.py`, ...). Deleting it would break the published package. |
|
|
25
|
+
| Managed Agents | **NOT SUBSTANTIATED** | The claimed "NOT TESTED against live Anthropic API" marker in SKILL.md was not found. The premise of the delete recommendation could not be reproduced. |
|
|
26
|
+
| "41 static roles" | **MISCHARACTERIZED** | `references/agent-types.md:9` already documents these precisely: "prompt-defined specifications the orchestrator adopts per phase, **not separate processes**", with "typically 5-10 for simple projects" activated. They are a catalog, not 41 running agents. There is no static-role runtime cost to delete. |
|
|
27
|
+
|
|
28
|
+
## Conclusion
|
|
29
|
+
|
|
30
|
+
**One of five candidates survived.** Four were live, mis-pathed, unsubstantiated,
|
|
31
|
+
or mischaracterized. This is consistent with the pattern across this whole
|
|
32
|
+
planning pass: the research corpus's confident specifics do not survive contact
|
|
33
|
+
with the repo, and the plan's "audit before deleting anything" rule was load
|
|
34
|
+
bearing rather than cautious.
|
|
35
|
+
|
|
36
|
+
`autonomy/voice.sh` is a genuine 501-line orphan. Even so, **no deletion is
|
|
37
|
+
performed here** per the founder decision. Recommended follow-up, separately and
|
|
38
|
+
evidence-gated: either delete `voice.sh` and keep the honest `cmd_voice` stub, or
|
|
39
|
+
keep the script if issue #85 will build on it. That is a small, isolated call
|
|
40
|
+
with no dependents by definition.
|
|
41
|
+
|
|
42
|
+
## What this does NOT claim
|
|
43
|
+
|
|
44
|
+
This audit covers only the five named candidates. It is not a repo-wide dead-code
|
|
45
|
+
sweep, and it makes no claim about the rest of the tree.
|
|
@@ -0,0 +1,137 @@
|
|
|
1
|
+
# v8 Major Release - Costed Decision Plan
|
|
2
|
+
|
|
3
|
+
Status: PROPOSAL (founder review before any implementation spend). 2026-07-24.
|
|
4
|
+
Author: this session, grounded in the actual repo state (not aspirational).
|
|
5
|
+
|
|
6
|
+
## TL;DR (the honest headline)
|
|
7
|
+
|
|
8
|
+
**v8 is ~80% built and tested, but uncommitted.** The hard engineering - a 12,300-line
|
|
9
|
+
parity-locked TypeScript runner, the Anthropic SDK route (`sdk_invoker.ts`, `sdk_mode.ts`,
|
|
10
|
+
stream parser), and ~18,000 lines of *working, test-passing* feature work (supervised builds,
|
|
11
|
+
deadline confinement, honesty/no-mock hardening, code-review calibration) - already exists in
|
|
12
|
+
the working tree. The V8-AGENT-SDK-PLAN and RARV-C-100X-PLAN both say so explicitly:
|
|
13
|
+
"much of this is already shipped."
|
|
14
|
+
|
|
15
|
+
So "the v8 major release" is NOT a build-from-scratch effort. It is, in order of cost:
|
|
16
|
+
1. **CONSOLIDATE + VERIFY + SHIP** what already works (cheap, high-value, the bulk of the win).
|
|
17
|
+
2. **A bounded net-new layer** the founder asked for: jcode-informed harness-intelligence,
|
|
18
|
+
PostHog analytics, the otel 2.x upgrade.
|
|
19
|
+
|
|
20
|
+
This plan costs each phase so the founder funds them selectively, not as one open-ended blob.
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## Where v8 actually stands (verified this session)
|
|
25
|
+
|
|
26
|
+
| Component | State | Evidence |
|
|
27
|
+
|---|---|---|
|
|
28
|
+
| TS runner (autonomous loop, council, gates, prompt) | Built, parity-locked | `loki-ts/src/runner/` 12.3k lines, 22 modules |
|
|
29
|
+
| Anthropic SDK route (judges = raw SDK, loop = Agent SDK) | Built + tested | `sdk_invoker.ts`, `sdk_mode.ts`, `sdk_stream_parser.ts`; tests: sdk_invoker, sdk_stream_parser, sdk_query_provider, sdk_loop_e2e |
|
|
30
|
+
| One-switch `LOKI_SDK_MODE` (off/judges/full) + rollback | Built | CHANGELOG v8.1; byte-mirrored bash+TS, shared parity fixture |
|
|
31
|
+
| Supervised builds / deadline confinement | Built + tests PASS | `deadline.py`, `build_supervisor.py`, `dependency-setup.sh` + `test-hard-deadline-confinement`, `test-supervised-*` (verified passing) |
|
|
32
|
+
| Honesty / no-mock hardening | Built + tests PASS | `no_mock_scan.py`, `test-harness-false-green`, `test-honest-gate-status`, `test-nomock-data-render` (verified passing) |
|
|
33
|
+
| Code-review calibration | Built | `test-review-severity-calibration`, `-assurance-tail`, `-lockfile-context` + council/quality_gates mods |
|
|
34
|
+
| ~18k LOC uncommitted total | Working, not committed | `git diff`: 46 files +7173/-1296; 25 new files +11057 LOC |
|
|
35
|
+
| Published to npm as 8.x | NO | npm latest = 7.129.4; v8 never published, never merged to main |
|
|
36
|
+
| PostHog analytics | **BUILT** (`5dcc88c6`) | opt-in `build_verified` event behind a strict second gate (`LOKI_ANALYTICS=on`, default OFF even when telemetry is on), fixed allowlist reader emits proof scalars only |
|
|
37
|
+
| otel 2.x (the deferred CVE fix) | **DONE** (`159dc7f4`) | clears GHSA-45rx-2jwx-cxfr at source |
|
|
38
|
+
| Phase 3 harness intelligence | **BUILT** (`9e596e9a`, `889bd52a`, `500e74f6`) | 3a prompt-cache discipline already existed (verified, not rebuilt); 3b confidence-spike, 3c goal scoring, 3d smart retry landed 2026-07-25 |
|
|
39
|
+
| Release identity (VERSION vs CHANGELOG) | **RESOLVED** (`a749ee54`) | folded the drafted `v8.1.0` block into one `v8.0.0` entry; no 8.x was ever published, so a first-ever 8.x as 8.1.0 would have described a release history that does not exist |
|
|
40
|
+
|
|
41
|
+
**Releasability gate:** full `local-ci.sh` on the WIP tree - RESULT PENDING (running at write time;
|
|
42
|
+
this plan's Phase 1 cost depends on how clean it is).
|
|
43
|
+
|
|
44
|
+
---
|
|
45
|
+
|
|
46
|
+
## The CHANGELOG-vs-WIP question (must resolve first)
|
|
47
|
+
|
|
48
|
+
The `## v8.1.0 (unreleased)` CHANGELOG entry documents ONLY the SDK one-switch/rollback/packaging
|
|
49
|
+
gate. But the ~18k uncommitted lines are mostly OTHER efforts (supervised builds, deadline,
|
|
50
|
+
honesty, review calibration). **The documented release scope and the actual WIP do not match.**
|
|
51
|
+
|
|
52
|
+
Resolution needed from founder: is the v8 MAJOR release =
|
|
53
|
+
- (A) the SDK route ONLY (commit just the SDK-related files, ship a focused v8.0.0), OR
|
|
54
|
+
- (B) EVERYTHING in the tree (SDK + supervised + honesty + review work) as one big v8.0.0?
|
|
55
|
+
|
|
56
|
+
Recommendation: **(B)**, because the extra work is on-thesis (all deepens the trust moat), already
|
|
57
|
+
tested, and shipping it piecemeal means more release cycles. But it requires per-theme commit
|
|
58
|
+
curation (never one `git add -A`), so it costs more review time than (A). This is the single
|
|
59
|
+
biggest scope decision and it changes Phase 1's cost.
|
|
60
|
+
|
|
61
|
+
---
|
|
62
|
+
|
|
63
|
+
## Phased plan (each phase independently fundable)
|
|
64
|
+
|
|
65
|
+
### Phase 1 - CONSOLIDATE + SHIP what's built [CHEAPEST, HIGHEST VALUE]
|
|
66
|
+
The bulk of v8's value is already written. This phase makes it real.
|
|
67
|
+
- Resolve the scope question (A vs B above).
|
|
68
|
+
- Curate the WIP into coherent, themed commits (per-theme, individually staged - the same
|
|
69
|
+
isolation discipline used for the v7.129.4 doc-gen commit; NEVER `git add -A`).
|
|
70
|
+
- Get full local-ci green on the committed tree (prime deps + rebuild dist - the lesson from
|
|
71
|
+
v7.129.4: most "failures" are environmental).
|
|
72
|
+
- Reconcile the release identity: VERSION 8.0.0 vs CHANGELOG's 8.1.0; first-ever 8.x publish; a
|
|
73
|
+
genuine MAJOR. Decide the number, write the CHANGELOG major entry.
|
|
74
|
+
- **Do NOT publish yet** - land it green + reviewed on the branch, PR-ready. Publishing 8.0.0 is a
|
|
75
|
+
separate, explicit, irreversible founder go (like the v7.129.4 push).
|
|
76
|
+
- Cost: MODERATE (review/curation of 18k LOC + CI greening). Mostly verification, little new code.
|
|
77
|
+
- Deliverable: v8 branch green, committed, coherent, PR-ready. The trust moat + SDK route SHIPPED.
|
|
78
|
+
|
|
79
|
+
### Phase 2 - PostHog analytics [NET-NEW, founder-requested]
|
|
80
|
+
"log/monitor/get analytics as much as possible in PostHog."
|
|
81
|
+
- Wire a PostHog sink alongside the existing OTLP/crash telemetry (opt-in, privacy-preserving -
|
|
82
|
+
loki's zero-egress posture is a documented moat; PostHog must be explicit-opt-in, never covert).
|
|
83
|
+
- The SDK route already surfaces rich typed telemetry (total_cost_usd, usage, cache read/creation,
|
|
84
|
+
modelUsage, num_turns, duration_ms) - these become PostHog events cheaply.
|
|
85
|
+
- Instrument: build lifecycle, gate outcomes, RARV iterations, provider/model, cost/tokens, council
|
|
86
|
+
verdicts, receipt facts, per-stage latency, failure modes.
|
|
87
|
+
- Cost: MODERATE (one new sink + event mapping; reuses existing telemetry seams).
|
|
88
|
+
- Deliverable: opt-in PostHog analytics across the v8 build lifecycle.
|
|
89
|
+
|
|
90
|
+
### Phase 3 - jcode-informed harness intelligence [NET-NEW, "better than jcode"]
|
|
91
|
+
Adopt jcode's measured-harness discipline onto loki's trust moat (see the jcode analysis in
|
|
92
|
+
[[project-v8-major-release-mandate]]). Ranked by impact/effort:
|
|
93
|
+
- 3a. **Prompt-cache discipline** (HIGH impact on latency+cost, MED effort): append-only context,
|
|
94
|
+
stable prefix, MCP tools advertised up-front. The SDK route makes cache metrics visible already.
|
|
95
|
+
- 3b. **Confidence-stepping / spike re-check** (HIGH, MED): force re-verify when agent confidence
|
|
96
|
+
jumps to 100 (jcode's finding: confident-after is not signal). Layers onto the completion council.
|
|
97
|
+
- 3c. **Hill-climbable goal scoring** (MED, MED): score each goal's quantifiability, push back on
|
|
98
|
+
un-measurable goals. Complements the evidence gate.
|
|
99
|
+
- 3d. **Auto-poke persistence** (MED, LOW): poke the loop back on incomplete todos; smart retry
|
|
100
|
+
(transient retry, non-retryable stop-to-save-tokens). Partially exists (never-stop directive).
|
|
101
|
+
- Cost: LARGE (each is a real feature). Fund individually; 3a is the best first (direct latency/cost win).
|
|
102
|
+
- Deliverable: measurably faster/cheaper/more-persistent loop, trust moat intact.
|
|
103
|
+
|
|
104
|
+
### Phase 4 - otel 2.x + dependency modernization [DEFERRED FROM v7.129.4]
|
|
105
|
+
- The breaking @opentelemetry 2.x major bump that clears GHSA-45rx-2jwx-cxfr for real (the v7.129.4
|
|
106
|
+
patch waived it as not-reachable; here it gets genuinely fixed since major bumps belong in a major).
|
|
107
|
+
- Re-verify tracing still works after the bump.
|
|
108
|
+
- Founder standing directive: "always latest code, api, libraries" - audit + modernize other deps here.
|
|
109
|
+
- Cost: SMALL-MODERATE (mechanical but needs tracing re-verification).
|
|
110
|
+
|
|
111
|
+
### Phase 5 - MAJOR RELEASE [IRREVERSIBLE, explicit founder go]
|
|
112
|
+
- Full CLAUDE.md release workflow: 14-file version bump, CHANGELOG major entry, dashboard rebuild,
|
|
113
|
+
pre-publish validation, merge to main -> publish npm/Docker/Homebrew.
|
|
114
|
+
- Post-release validation across all channels + both routes (as done for v7.129.4).
|
|
115
|
+
- Cost: SMALL (mechanical, proven process from v7.129.4). The GO is the founder's.
|
|
116
|
+
|
|
117
|
+
---
|
|
118
|
+
|
|
119
|
+
## Recommended sequence + why
|
|
120
|
+
|
|
121
|
+
1. **Phase 1 first, always.** It converts existing work into a shipped, reviewed, green branch -
|
|
122
|
+
the highest value for the least new spend, and it de-risks everything else.
|
|
123
|
+
2. Then the founder picks Phase 2 (PostHog) and/or Phase 3 items (harness intelligence) to fund -
|
|
124
|
+
these are the "better in every direction" asks and each is independently costed.
|
|
125
|
+
3. Phase 4 (otel/deps) bundles into whichever phase touches CI next.
|
|
126
|
+
4. Phase 5 (publish) only when the founder says the accumulated phases are worth a major release.
|
|
127
|
+
|
|
128
|
+
## Cost discipline (per the standing directives)
|
|
129
|
+
- Ultrathink each phase's plan + advisor review before its implementation.
|
|
130
|
+
- Ultracode (surgical workflow) for the implementation where parallelism pays - NOT for planning.
|
|
131
|
+
- Every phase ends in a SHIPPED/committed/verified artifact, never another doc.
|
|
132
|
+
- No `git add -A`; per-theme individual staging; repo-local asklokesh identity; explicit go before
|
|
133
|
+
any irreversible publish.
|
|
134
|
+
|
|
135
|
+
## The one thing to decide now
|
|
136
|
+
The founder's call that unblocks everything: **the CHANGELOG-vs-WIP scope question (A vs B) + confirm
|
|
137
|
+
Phase 1 as the start.** Everything downstream flows from that.
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
# v8 overnight execution plan (2026-07-25)
|
|
2
|
+
|
|
3
|
+
Autonomous session. Founder asleep ~8h. Standing authorization: make the
|
|
4
|
+
architectural calls, validate everything, commit and push verified increments.
|
|
5
|
+
|
|
6
|
+
## Governing principles
|
|
7
|
+
|
|
8
|
+
1. **Already-exists check FIRST on every item.** It has been right 5 times out
|
|
9
|
+
of 5 this session (evidence-in-PR, offline verifier, stuck signals, signing,
|
|
10
|
+
fail-closed SDK). The research corpus is consistently more confident than
|
|
11
|
+
correct.
|
|
12
|
+
2. **"Don't overwhelm users" is an architectural constraint, not a preference.**
|
|
13
|
+
Every new env var is user-facing complexity. Prefer: safe default, honest
|
|
14
|
+
degradation, structured event for operators, NO new knob. If the safe
|
|
15
|
+
behavior can be the default, make it the default.
|
|
16
|
+
3. **Never mark done what is not verified.** A test that passes against the
|
|
17
|
+
pre-change code proves nothing; every behavioral change gets the
|
|
18
|
+
revert-and-confirm-it-fails treatment.
|
|
19
|
+
4. **Gate on the CI verdict line, never the exit code.**
|
|
20
|
+
5. **`bun run build` after every `loki-ts/src` edit** - the shipped package runs
|
|
21
|
+
`dist`, not `src`.
|
|
22
|
+
|
|
23
|
+
## Order (by reversibility, not task number)
|
|
24
|
+
|
|
25
|
+
| # | Item | Shape | Status |
|
|
26
|
+
|---|---|---|---|
|
|
27
|
+
| 11 | SDK capability-degradation event | small increment | premise mostly satisfied |
|
|
28
|
+
| 10 | Phase-0 runtime-truth audit | docs from source | gates 12 |
|
|
29
|
+
| 12 | Flip SDK default | TESTS first, flip only if green | conditional |
|
|
30
|
+
| 18 | Typed run journal | MEASURE first | may collapse |
|
|
31
|
+
| 20 | First-4-min preview | MEASURE first | may collapse |
|
|
32
|
+
| 17 | Workspace isolation | interface + Local adapter | Docker = design only |
|
|
33
|
+
| 23 | Helm/RBAC/SSO | real, testable offline | helm lint / kubeval |
|
|
34
|
+
| 15 | npm SDK | build fully, DO NOT PUBLISH | founder gate |
|
|
35
|
+
| 22 | Managed cloud | design doc only | blocked on P0 results |
|
|
36
|
+
| 24 | SWE-bench RESOLUTION | DO NOT RUN | ~$950 spend, deferred |
|
|
37
|
+
|
|
38
|
+
## Hard stops (will NOT do autonomously)
|
|
39
|
+
|
|
40
|
+
- **#15 npm publish.** Building the package is in scope; `npm publish` is
|
|
41
|
+
outward-facing and irreversible. "Commit and push" authorized the feature
|
|
42
|
+
branch, not the public registry. The publish command is left for the founder.
|
|
43
|
+
- **#24 SWE-bench RESOLUTION run.** 119 pinned instances at
|
|
44
|
+
`LOKI_BUDGET_LIMIT=8` is up to ~$950 of real spend, on a benchmark the
|
|
45
|
+
approved plan already marks DEFERRED by founder decision.
|
|
46
|
+
- **Merge to main / publish 8.0.0.** Standing hard founder gate.
|
|
47
|
+
|
|
48
|
+
## #12 authorization, stated precisely
|
|
49
|
+
|
|
50
|
+
The approved plan says: do not flip until parity AND recovery tests pass. That
|
|
51
|
+
is a CONDITIONAL authorization, so #12's real content is writing those tests
|
|
52
|
+
(acceptance #1 SDK-full with the claude binary absent, #7 SIGKILL recovery, #8
|
|
53
|
+
resume does not repeat irreversible actions, plus both-route parity). If they
|
|
54
|
+
pass, flipping executes the founder's decision. If any fail, do not flip and
|
|
55
|
+
name the blocker.
|
|
56
|
+
|
|
57
|
+
## Findings that reshaped the queue
|
|
58
|
+
|
|
59
|
+
**#11 is nearly done already.** `autonomous.ts:356 requireModule()` is fail-fast
|
|
60
|
+
(a load-bearing module that cannot load is a fatal contract violation, not a
|
|
61
|
+
silent stub). `providers.ts:559-640` is fail-closed: no CLI fallback,
|
|
62
|
+
`exitCode=1` on throw, and `sawResult ? res.exitCode : 1` so a stream that never
|
|
63
|
+
produced a terminal result cannot be counted as success. The SDK path's MAIN
|
|
64
|
+
LOOP has no `claude`-binary dependency.
|
|
65
|
+
|
|
66
|
+
**Scope stated exactly, 2026-07-26.** `providers.ts:570` delegates every
|
|
67
|
+
non-mainLoop call back to `claudeProvider()`, which reaches the binary via
|
|
68
|
+
`ensureClaudeHelpCache()`. But that is the `LOKI_SDK_MODE=off` path, and
|
|
69
|
+
reading it alone to mean "the judge path shells out to claude" is wrong:
|
|
70
|
+
under `judges`/`full`, `sdkModeDefaults()` enables all seven `JUDGE_VARS` and
|
|
71
|
+
those sites have their own raw-SDK bridges. `completion-council.sh:2910` says
|
|
72
|
+
so directly -- the raw-SDK vote path "needs no claude binary" (its precondition
|
|
73
|
+
is bun plus the bridge), falling closed to `claude` only on an SDK miss.
|
|
74
|
+
|
|
75
|
+
So under `LOKI_SDK_MODE=full` BOTH the loop and the judges have binary-free
|
|
76
|
+
paths; the residual CLI dependency is off-mode delegation plus the fail-closed
|
|
77
|
+
fallback. Pinned by
|
|
78
|
+
`loki-ts/tests/runner/acceptance_sdk_binary_absent.test.ts`.
|
|
79
|
+
|
|
80
|
+
The genuine remaining gap is OBSERVABILITY: an SDK load/stream failure is
|
|
81
|
+
written into captured text, not emitted as a structured capability-degradation
|
|
82
|
+
event an operator can alert on. That is the increment.
|
|
@@ -0,0 +1,232 @@
|
|
|
1
|
+
# V8 Runtime Truth Audit (Phase 0)
|
|
2
|
+
|
|
3
|
+
Date: 2026-07-25. Source-verified. Gates the SDK-default flip (task #12).
|
|
4
|
+
|
|
5
|
+
Method: read the actual code paths, not the comments describing them. Where a
|
|
6
|
+
claim concerns the Agent SDK's own capabilities, it is checked against the
|
|
7
|
+
primary docs (code.claude.com), not inferred.
|
|
8
|
+
|
|
9
|
+
## Executive summary
|
|
10
|
+
|
|
11
|
+
The v8 SDK route is **better engineered than the competitive research assumed**
|
|
12
|
+
on the two properties that matter most for trust. After the section-4 correction
|
|
13
|
+
(2026-07-25) **no known capability regression blocks the flip** - what remains
|
|
14
|
+
is three untested acceptance items. The gating work is TESTS, not a port.
|
|
15
|
+
|
|
16
|
+
| Property | Status |
|
|
17
|
+
|---|---|
|
|
18
|
+
| Fail-closed on error | **SOUND** |
|
|
19
|
+
| No silent CLI fallback | **SOUND** |
|
|
20
|
+
| Module-resolution fail-fast | **SOUND** |
|
|
21
|
+
| Rollback escape hatch | **SOUND** |
|
|
22
|
+
| Structured degradation event | **ADDED 2026-07-25** |
|
|
23
|
+
| Stagnation / done-signal valves | **PORTED 2026-07-24** |
|
|
24
|
+
| Session continuity across iterations | **NOT A GAP** (corrected 2026-07-25; opt-in recovery parity only) |
|
|
25
|
+
| Acceptance #1 / #7 / #8 | **TESTED 2026-07-26** (28 assertions, each with a verified negative control) |
|
|
26
|
+
|
|
27
|
+
## 1. Route resolution (verified)
|
|
28
|
+
|
|
29
|
+
`selectClaudeInvokerKind()` (`loki-ts/src/runner/providers.ts`) is pure over env,
|
|
30
|
+
so the rollback path is unit-testable without spawning a process:
|
|
31
|
+
|
|
32
|
+
1. `LOKI_LEGACY_BASH` truthy -> `legacy`. **The rollback always wins**, even if
|
|
33
|
+
the loop is opted-on or later becomes default-on.
|
|
34
|
+
2. else `LOKI_SDK_LOOP` truthy -> `sdk`.
|
|
35
|
+
3. else -> `legacy` (default-off; byte-identical to v8 as shipped).
|
|
36
|
+
|
|
37
|
+
`LOKI_SDK_MODE` normalizes to `off|judges|full` on both routes with a fail-safe:
|
|
38
|
+
an unknown value falls back to `off`, so the SDK can never be switched on by a
|
|
39
|
+
typo. Mirrored byte-for-byte in `autonomy/lib/sdk-mode.sh` and
|
|
40
|
+
`loki-ts/src/runner/sdk_mode.ts`.
|
|
41
|
+
|
|
42
|
+
## 2. Fail-closed behavior (verified, and stronger than claimed)
|
|
43
|
+
|
|
44
|
+
The research asserted the SDK path might "silently fall back to the CLI". It
|
|
45
|
+
does not. In `providers.ts`:
|
|
46
|
+
|
|
47
|
+
- The SDK is loaded by a **lazy dynamic import**, so the default-off path never
|
|
48
|
+
pays to load it.
|
|
49
|
+
- A load failure (SDK missing, platform binary absent) sets `exitCode = 1`.
|
|
50
|
+
There is no CLI fallback branch to take.
|
|
51
|
+
- A thrown `query()` is caught, appends the error to captured text, and leaves
|
|
52
|
+
`exitCode = 1`.
|
|
53
|
+
- **`exitCode = res.sawResult ? res.exitCode : 1`** - a stream that never
|
|
54
|
+
produced a terminal result is a FAILED iteration, never counted as success.
|
|
55
|
+
This is the property that stops a broken SDK run from being read as a green
|
|
56
|
+
build.
|
|
57
|
+
|
|
58
|
+
Module resolution is equally strict. `requireModule()`
|
|
59
|
+
(`loki-ts/src/runner/autonomous.ts:356`) throws on an unloadable helper rather
|
|
60
|
+
than substituting a stub, and the surrounding comment documents why the previous
|
|
61
|
+
stubs were removed: they "degraded SILENTLY to WRONG results rather than safe
|
|
62
|
+
no-ops".
|
|
63
|
+
|
|
64
|
+
**Conclusion: acceptance item #3 (no silent SDK-full fallback to legacy) is
|
|
65
|
+
SATISFIED in behavior.** What was missing was observability, addressed below.
|
|
66
|
+
|
|
67
|
+
## 3. Capability degradation is now observable (added this session)
|
|
68
|
+
|
|
69
|
+
Before: an SDK load or stream failure existed only as prose inside the captured
|
|
70
|
+
output. An operator running unattended had nothing to alert on, and no way to
|
|
71
|
+
distinguish "the SDK could not load" from "the model did poor work".
|
|
72
|
+
|
|
73
|
+
Now: `emitSdkDegradationEvent()` appends a structured record to the SAME
|
|
74
|
+
append-only `.loki/events.jsonl` stream the hook events use, with the same
|
|
75
|
+
`{type, source, timestamp, payload}` envelope, so every existing consumer picks
|
|
76
|
+
it up for free:
|
|
77
|
+
|
|
78
|
+
```json
|
|
79
|
+
{"type":"capability_degraded","source":"sdk_loop","timestamp":"...",
|
|
80
|
+
"payload":{"capability":"sdk_query","fail_closed":true,"reason":"...",
|
|
81
|
+
"tier":"...","model":"...","iteration":"..."}}
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
`fail_closed: true` is stated in the record rather than left for a reader to
|
|
85
|
+
assume. Deliberately **no new env var**: this is signal an operator always
|
|
86
|
+
wants, and a knob to enable your own error reporting is a knob nobody finds.
|
|
87
|
+
Guarded by `loki-ts/tests/runner/sdk_degradation_event.test.ts` (5 tests),
|
|
88
|
+
including the load-bearing property that it NEVER throws - a diagnostic that
|
|
89
|
+
breaks the run it describes is worse than no diagnostic.
|
|
90
|
+
|
|
91
|
+
## 4. Session continuity: an OPT-IN recovery gap, NOT a default regression
|
|
92
|
+
|
|
93
|
+
**CORRECTED 2026-07-25.** The first version of this section called this "THE GAP
|
|
94
|
+
that blocks the flip" and claimed the default route "would silently lose
|
|
95
|
+
cross-iteration context that the legacy route preserves." **That claim was
|
|
96
|
+
false**, and it is recorded here rather than quietly deleted because the way it
|
|
97
|
+
was wrong is the same trap this audit exists to catch: it asserted a parity gap
|
|
98
|
+
without first checking whether the legacy side was even turned on.
|
|
99
|
+
|
|
100
|
+
What the source actually says:
|
|
101
|
+
|
|
102
|
+
- `sessionStampEnabled()` returns false unless `LOKI_SESSION_STAMP=1`
|
|
103
|
+
(`claude_flags.ts:378`). **Default OFF.**
|
|
104
|
+
- `resumeSessionEnabled()` returns false unless `LOKI_RESUME_SESSION=1`
|
|
105
|
+
(`claude_flags.ts:452`). **Default OFF.**
|
|
106
|
+
- No shipped default sets either to `1` (grep across `*.sh`, `*.ts`, `*.json`,
|
|
107
|
+
`Dockerfile*`, excluding tests: only definitions and comments).
|
|
108
|
+
- Decisively: `sessionStampArgv()` emits
|
|
109
|
+
`claudeIterationSessionUuid()` = `uuidv5("${runId}:${iteration}")`
|
|
110
|
+
(`claude_flags.ts:363-372`) - **derived from the iteration number, so a
|
|
111
|
+
DISTINCT id every iteration.** `tests/test-cli-session-v734.sh:11` states the
|
|
112
|
+
same in its own words: "a PER-ITERATION DISTINCT" session id.
|
|
113
|
+
|
|
114
|
+
A per-iteration distinct `--session-id` is **correlation/tracing metadata, not
|
|
115
|
+
conversation continuity.** It gives each iteration its own session rather than
|
|
116
|
+
threading one. And `sessionResumeArgv()` is documented at `claude_flags.ts:445`
|
|
117
|
+
as "Recovery only, never a per-iteration chain" - one resume on the first call
|
|
118
|
+
of a restarted run.
|
|
119
|
+
|
|
120
|
+
**Therefore: on the default path the legacy `claude -p` route is ALSO stateless
|
|
121
|
+
per iteration. The SDK route loses nothing by default, and flipping it does not
|
|
122
|
+
regress cross-iteration context.**
|
|
123
|
+
|
|
124
|
+
The real, much smaller gap: **under opt-in `LOKI_RESUME_SESSION=1`, legacy
|
|
125
|
+
performs one recovery resume after a restart and the SDK path does nothing.**
|
|
126
|
+
That is opt-in recovery parity, not a default-path capability regression. It
|
|
127
|
+
does not block the flip; it is a follow-up for anyone using that knob.
|
|
128
|
+
|
|
129
|
+
The platform supports it whenever that port is done. Per the primary Agent SDK
|
|
130
|
+
docs (code.claude.com/docs/en/agent-sdk/sessions), `query()` accepts
|
|
131
|
+
`resume: <session_id>`, `forkSession: true`, and `continue: true`; session ids
|
|
132
|
+
are readable from `SDKResultMessage.session_id`.
|
|
133
|
+
|
|
134
|
+
Two traps for that future port, both load-bearing:
|
|
135
|
+
|
|
136
|
+
1. **Do not reuse `resumeSessionEnabled()` on the SDK path.** It ends in
|
|
137
|
+
`claudeFlagSupported("--resume")`, which probes the `claude` **binary**.
|
|
138
|
+
Acceptance #1 is "SDK-full works with the binary absent" - reusing that
|
|
139
|
+
predicate makes resume silently unavailable in precisely the scenario the SDK
|
|
140
|
+
exists for. The SDK path needs the env check WITHOUT the CLI probe.
|
|
141
|
+
2. **Do not write an SDK `session_id` into `.loki/state/claude-session.json`.**
|
|
142
|
+
`resumeTargetUuid()` reads `claude_session_uuid` and regex-validates it as a
|
|
143
|
+
CLI-stamped uuid; an SDK session id is a different id space from a different
|
|
144
|
+
mechanism. Sharing the field lets a restart hand one route the other's id.
|
|
145
|
+
|
|
146
|
+
**And the cwd caveat** (docs, verbatim): a `resume` call from a different
|
|
147
|
+
directory looks in the wrong place and silently returns a FRESH session rather
|
|
148
|
+
than erroring. Any port must pin cwd and assert the resumed id EQUALS the stored
|
|
149
|
+
one - a test that only checks "the call did not throw" passes against a port
|
|
150
|
+
that starts fresh every time.
|
|
151
|
+
|
|
152
|
+
## 5. Flip prerequisites (task #12)
|
|
153
|
+
|
|
154
|
+
The approved plan authorizes the flip conditionally: only once parity AND
|
|
155
|
+
recovery tests pass. Current state:
|
|
156
|
+
|
|
157
|
+
| Prerequisite | Status |
|
|
158
|
+
|---|---|
|
|
159
|
+
| Stagnation + done-signal valves on TS route | **DONE** (2026-07-24, 10 tests, 9 fail against the pre-port stub) |
|
|
160
|
+
| No silent fallback | **DONE** (verified sound; degradation event added) |
|
|
161
|
+
| Session continuity parity | **NOT A BLOCKER** (corrected - see section 4; both legacy knobs default OFF and the stamp is per-iteration distinct, so there is no default-path continuity to regress) |
|
|
162
|
+
| Acceptance #1 (SDK-full works with `claude` binary absent) | **DONE 2026-07-26** - `loki-ts/tests/runner/acceptance_sdk_binary_absent.test.ts` (9). Scope corrected below. |
|
|
163
|
+
| Acceptance #7 (SIGKILL recoverable without corruption) | **DONE 2026-07-26** - `loki-ts/tests/runner/acceptance_sigkill_recovery.test.ts` (10). |
|
|
164
|
+
| Acceptance #8 (resume does not repeat irreversible actions) | **DONE 2026-07-26** - `tests/test-acceptance-resume-idempotence.sh` (9, incl. a mutation check). **Naming collision** flagged at `claude_flags.ts:438`: Loki's own checkpoint `--resume` is a different layer from the claude-CLI session resume. #8 is the CHECKPOINT layer, so it was testable now and was never downstream of session continuity. |
|
|
165
|
+
|
|
166
|
+
### What the three test files established (2026-07-26)
|
|
167
|
+
|
|
168
|
+
Each has a **verified negative control** - the implementation was temporarily
|
|
169
|
+
broken and the test confirmed RED before being committed - because all three
|
|
170
|
+
premises already looked satisfied in source, which is precisely the condition
|
|
171
|
+
under which a new test passes vacuously and proves nothing.
|
|
172
|
+
|
|
173
|
+
- **#1, with the scope stated exactly.** The main agentic loop is genuinely
|
|
174
|
+
binary-free: route resolution is pure over env, the SDK loads by lazy dynamic
|
|
175
|
+
import, and no spawn/shell/flag-probe appears anywhere in
|
|
176
|
+
`sdkQueryProvider`'s own body.
|
|
177
|
+
|
|
178
|
+
**Two corrections, in opposite directions, both recorded rather than
|
|
179
|
+
quietly fixed.** The `providers.ts:570` delegation IS real: every non-mainLoop
|
|
180
|
+
call goes back to `claudeProvider()`, which reaches the binary via
|
|
181
|
+
`ensureClaudeHelpCache()`. But reading that line alone and concluding "the
|
|
182
|
+
judge path shells out to claude" was ALSO wrong, in the same way section 4's
|
|
183
|
+
original error was wrong: it described one route while a second, enabled by a
|
|
184
|
+
different switch, was sitting next to it.
|
|
185
|
+
|
|
186
|
+
`:570` is the **`LOKI_SDK_MODE=off` path**. Under `judges` or `full`,
|
|
187
|
+
`sdkModeDefaults()` turns on all seven `JUDGE_VARS`
|
|
188
|
+
(`LOKI_SDK_COUNCIL_VOTE`, `LOKI_SDK_CODE_REVIEW`, `LOKI_SDK_DONE_RECOG`,
|
|
189
|
+
`LOKI_SDK_COUNCIL_V2`, `LOKI_SDK_VOTER_AGENTS`, `LOKI_SDK_GRILL`,
|
|
190
|
+
`LOKI_SDK_PRD_ENRICH`), and those sites have their own raw-SDK bridges.
|
|
191
|
+
`completion-council.sh:2910` states it directly: the raw-SDK vote path
|
|
192
|
+
"needs no claude binary", and the precondition check is satisfied by
|
|
193
|
+
bun plus the bridge instead. It falls closed to `claude` only on an SDK miss.
|
|
194
|
+
|
|
195
|
+
**So: under `LOKI_SDK_MODE=full` both the loop and the judges have
|
|
196
|
+
binary-free paths.** The residual CLI dependency is the `off`-mode
|
|
197
|
+
delegation and the fail-closed fallback, not a hole in SDK-full. The
|
|
198
|
+
delegation boundary is pinned by a test so a future edit cannot silently
|
|
199
|
+
redefine what "SDK-full" means in either direction.
|
|
200
|
+
- **#7.** Truncated state is reported `corrupted: true`, never as a clean
|
|
201
|
+
start, and the bad file is preserved for forensics - the same
|
|
202
|
+
empty-vs-invalid trap as the v7.129.5 receipt bug. A state left at `running`
|
|
203
|
+
is correctly treated as a crash and reset rather than resumed, while the
|
|
204
|
+
genuinely resumable statuses (`paused`, `interrupted`, `budget_exceeded`,
|
|
205
|
+
`stopped`) keep their counters. Orphaned `*.tmp.*` files are swept, and a
|
|
206
|
+
RECENT tmp file is deliberately left alone so the sweep cannot race a live
|
|
207
|
+
writer. No test kills a real process: the states a SIGKILL produces are
|
|
208
|
+
constructed directly, so nothing depends on winning a race.
|
|
209
|
+
- **#8.** The PR guard keys on REMOTE state (`gh pr list --head --state open`),
|
|
210
|
+
not a local marker - so it survives the case that matters, a resume on a
|
|
211
|
+
fresh container where local `.loki/` is gone. The test simulates exactly that
|
|
212
|
+
wipe and asserts exactly one `gh pr create` across two completion passes;
|
|
213
|
+
the mutation check removes the guard and confirms a duplicate PR appears.
|
|
214
|
+
|
|
215
|
+
**Recommendation: the test prerequisites for the flip are now MET.** All three
|
|
216
|
+
acceptance items are covered with non-vacuous tests, the valves are ported, and
|
|
217
|
+
no known capability regression remains. The one scope correction above (judges
|
|
218
|
+
still take the CLI route) does not block the flip, because the flip changes the
|
|
219
|
+
main loop only - but it does mean "SDK-full runs with no `claude` binary
|
|
220
|
+
installed" is not yet true end-to-end, and should not be claimed in user-facing
|
|
221
|
+
docs until the judge path is ported too.
|
|
222
|
+
|
|
223
|
+
The flip itself remains a one-line change whose safety is entirely supplied by
|
|
224
|
+
the tests around it, and is left as a deliberate, separately-reviewable commit.
|
|
225
|
+
|
|
226
|
+
## 6. What this audit deliberately does not claim
|
|
227
|
+
|
|
228
|
+
This covers the claude-provider SDK path, route resolution, fail-closed
|
|
229
|
+
semantics, and session continuity. It does NOT cover: the judge/subcall path in
|
|
230
|
+
detail, MCP tool loading under SDK-full, budget/effort propagation, or the
|
|
231
|
+
Codex/Cline/Aider adapters. Those remain untriaged and are the natural next
|
|
232
|
+
slice.
|
|
@@ -0,0 +1,129 @@
|
|
|
1
|
+
# Claude Agent SDK Research (TypeScript) - for the v8 Loki migration
|
|
2
|
+
|
|
3
|
+
Sources: live docs `code.claude.com/docs/en/agent-sdk` + `/agent-sdk/typescript` (WebFetch, today); installed `claude` v2.1.207 `--help`; and the **actual unpacked `@anthropic-ai/claude-agent-sdk@0.3.207` tarball** (published 3 days ago) - `sdk.d.ts` (6923 lines), `package.json`, `sdk.mjs`. Every API shape below was verified against the real `.d.ts`, not just docs (WebFetch invented a few types - those are flagged UNVERIFIED). Package repo: `github.com/anthropics/claude-agent-sdk-typescript`.
|
|
4
|
+
|
|
5
|
+
## THE ONE FINDING THAT REFRAMES THE FOUNDER'S PREMISE (verified, load-bearing)
|
|
6
|
+
|
|
7
|
+
**The Claude Agent SDK does NOT remove the `claude` binary dependency. It IS the `claude` binary, wrapped as a subprocess.**
|
|
8
|
+
|
|
9
|
+
Primary-source evidence from the 0.3.207 tarball:
|
|
10
|
+
- `package.json`: `"dependencies": {}` but `"optionalDependencies"` ships **8 platform-specific native Claude Code binaries** (`@anthropic-ai/claude-agent-sdk-{linux,darwin,win32}-{x64,arm64}[-musl]@0.3.207`).
|
|
11
|
+
- `sdk.mjs` imports `node:child_process` and calls `spawn()` (2 call sites, 291 refs to `claude`, 13 to `claude-code`).
|
|
12
|
+
- `Options` exposes `executable?: 'bun'|'deno'|'node'`, `pathToClaudeCodeExecutable?: string`, `executableArgs`, and `spawnClaudeCodeProcess?: (o) => SpawnedProcess` - all subprocess-management knobs. There's even an `extractFromBunfs()` export whose only job is to locate the embedded `claude` binary when you `bun build --compile`.
|
|
13
|
+
- Docs confirm: *"The TypeScript SDK bundles a native Claude Code binary for your platform as an optional dependency."* README: *"The Claude Code SDK is now the Claude Agent SDK."* - it's a rename of the Claude Code SDK.
|
|
14
|
+
|
|
15
|
+
**Implication for the founder requirements.** Adopting `@anthropic-ai/claude-agent-sdk` gives Loki a first-class TS library, structured messages, in-process tools, and the full Claude Code harness - but it does **not** deliver the stated KEY strategic win (removing the CLI binary / "just an API key" containers). The binary is bundled instead of PATH-installed (so: no interactive login, no separate `claude` install step, no user-managed version drift - the SDK pins its own binary), but a native Claude Code executable still runs as a child process in every container. It's still "Claude Code," just vendored and spawned by your code instead of shelled out to on PATH.
|
|
16
|
+
|
|
17
|
+
If "NO claude binary at all, pure API calls" is a hard requirement, the surface that actually delivers it is **`@anthropic-ai/sdk` (the Client SDK, v0.111.0)** - pure HTTPS to `api.anthropic.com`, zero binary - but then Loki owns the agent loop, tools, context management, and permissions itself (which RARV-C/run.sh largely already does). Or **Managed Agents** (Anthropic hosts the loop + sandbox; REST only). This is the real architectural fork; recommend surfacing it to the founder before committing.
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## 1. Core primitives (verified against `sdk.d.ts`)
|
|
22
|
+
|
|
23
|
+
**Entry point - `query()` (sdk.d.ts:2527):**
|
|
24
|
+
```ts
|
|
25
|
+
export declare function query(_params: {
|
|
26
|
+
prompt: string | AsyncIterable<SDKUserMessage>;
|
|
27
|
+
options?: Options;
|
|
28
|
+
}): Query;
|
|
29
|
+
```
|
|
30
|
+
- `prompt: string` = one-shot. `prompt: AsyncIterable<SDKUserMessage>` = streaming-input mode (enables mid-session control methods).
|
|
31
|
+
- Returns a `Query` (below); you `for await (const message of query(...))` to drive the agentic loop. The SDK runs the whole Reason→Act tool-use loop internally - you don't write the `while stop_reason === 'tool_use'` loop (that's the explicit contrast the docs draw vs the Client SDK).
|
|
32
|
+
|
|
33
|
+
**`Query` interface (sdk.d.ts:2230) - `extends AsyncGenerator<SDKMessage, void>`** with control methods (all verified present):
|
|
34
|
+
`interrupt()`, `setPermissionMode(mode)`, `setModel(model?)`, `setMaxThinkingTokens(n, display?)`, `applyFlagSettings(settings)`, `supportedModels()`, `supportedAgents()`, `supportedCommands()`, `mcpServerStatus()`, `setMcpServers()`, `reconnectMcpServer()`, `toggleMcpServer()`, `rewindFiles(userMessageId, {dryRun?})`, `streamInput(stream)`, `initializationResult()`, `reinitialize()`, `close()`. Mid-session setters (`setModel`, `setPermissionMode`, `applyFlagSettings`) work **only in streaming-input mode**.
|
|
35
|
+
|
|
36
|
+
**Custom tools - `tool()` (sdk.d.ts:6745) + `createSdkMcpServer()` (sdk.d.ts:467):**
|
|
37
|
+
```ts
|
|
38
|
+
export declare function tool<Schema extends AnyZodRawShape>(
|
|
39
|
+
_name, _description, _inputSchema: Schema,
|
|
40
|
+
_handler: (args: InferShape<Schema>, extra) => Promise<CallToolResult>,
|
|
41
|
+
_extras?: { annotations?: ToolAnnotations }
|
|
42
|
+
): SdkMcpToolDefinition<Schema>;
|
|
43
|
+
|
|
44
|
+
export declare function createSdkMcpServer(_options: CreateSdkMcpServerOptions): McpSdkServerConfigWithInstance;
|
|
45
|
+
```
|
|
46
|
+
Tools are **in-process** (Zod-schema'd TS functions), bundled into an SDK MCP server, passed via `options.mcpServers`. This is how Loki would expose its own tools (queue, memory, council) to the agent without a separate MCP process - a genuine enhancement over shelling out.
|
|
47
|
+
|
|
48
|
+
**Built-in tools (no implementation needed):** Read, Write, Edit, Bash, Monitor, Glob, Grep, WebSearch, WebFetch, AskUserQuestion, Agent (subagent invocation), + MCP. Gated by `allowedTools`/`disallowedTools`/`tools`.
|
|
49
|
+
|
|
50
|
+
**Subagents / multi-agent - `options.agents: Record<string, AgentDefinition>` (sdk.d.ts:1327):**
|
|
51
|
+
```ts
|
|
52
|
+
type AgentDefinition = {
|
|
53
|
+
description: string; prompt: string;
|
|
54
|
+
tools?: string[]; disallowedTools?: string[];
|
|
55
|
+
model?: string; // alias, full id, or "inherit"
|
|
56
|
+
mcpServers?: AgentMcpServerSpec[]; skills?: string[];
|
|
57
|
+
maxTurns?: number; background?: boolean;
|
|
58
|
+
memory?: 'user'|'project'|'local';
|
|
59
|
+
effort?: 'low'|'medium'|'high'|'xhigh'|'max'|number;
|
|
60
|
+
permissionMode?: PermissionMode;
|
|
61
|
+
initialPrompt?: string; /* + experimental fields */
|
|
62
|
+
};
|
|
63
|
+
```
|
|
64
|
+
Subagents invoked via the `Agent` tool (include `"Agent"` in `allowedTools` to auto-approve). Messages from a subagent carry `parent_tool_use_id` so you can attribute them. This maps cleanly onto Loki's dev-fleet/council pattern - but note it's **one level deep** and runs subagents within the same host process, unlike Loki's current parallel-Task fan-out.
|
|
65
|
+
|
|
66
|
+
**Memory/context:** three layers. (a) `CLAUDE.md`/`.claude/` loaded via `settingSources` (`'user'|'project'|'local'`). (b) Automatic context compaction is built into the Claude Code harness the SDK spawns. (c) `AgentDefinition.memory` + Skills (`.claude/skills/*/SKILL.md`, `options.skills`). There is no separate vector-memory primitive - Loki's `memory/` Python package has no SDK equivalent; you'd wire it in as SDK MCP tools or hooks.
|
|
67
|
+
|
|
68
|
+
**Hooks:** `options.hooks` - `PreToolUse`, `PostToolUse`, `Stop`, `SessionStart`, `SessionEnd`, `UserPromptSubmit`, etc., as in-process callbacks (validate/block/transform/log). Good fit for Loki's quality gates and telemetry.
|
|
69
|
+
|
|
70
|
+
## 2. Session continuity (verified)
|
|
71
|
+
- `options.resume?: string` (session id), `options.continue?: boolean` (most recent), `options.forkSession?: boolean`, `options.resumeSessionAt?: string` (resume at a specific message UUID), `options.sessionId?: string` (pin a UUID), `options.persistSession?: boolean`.
|
|
72
|
+
- Session id comes from the `system`/`init` message (`message.session_id`) on the first query; pass it as `resume` on the next.
|
|
73
|
+
- Storage: **JSONL on your filesystem** (docs confirm; helpers `listSessions()`, `getSessionMessages()`, `getSessionInfo()`, `renameSession()`, `tagSession()`). `options.sessionStore` can mirror transcripts to an external backend.
|
|
74
|
+
- Maps to the CLI's `--resume`/`--session-id`/`--continue`/`--fork-session`/`--resume-session-at` (all present in `claude --help`). This is a strict superset of Loki's current CLI-flag resume usage.
|
|
75
|
+
|
|
76
|
+
## 3. Structured output (verified - direct CLI `--json-schema` equivalent)
|
|
77
|
+
```ts
|
|
78
|
+
outputFormat?: OutputFormat; // sdk.d.ts:1686
|
|
79
|
+
type OutputFormat = JsonSchemaOutputFormat; // :2016
|
|
80
|
+
type JsonSchemaOutputFormat = { type: 'json_schema'; schema: Record<string, unknown> }; // :898
|
|
81
|
+
```
|
|
82
|
+
Result lands on **`SDKResultSuccess.structured_output?: unknown`** (sdk.d.ts). There's a dedicated error subtype `error_max_structured_output_retries`, so the harness retries to satisfy the schema. This is the exact analog of the CLI's `--json-schema` flag. **Yes, fully supported.**
|
|
83
|
+
|
|
84
|
+
## 4. Budget / effort / model / fallback (verified - direct CLI-flag parity)
|
|
85
|
+
| Need | Agent SDK option | CLI flag |
|
|
86
|
+
|---|---|---|
|
|
87
|
+
| Budget cap | `maxBudgetUsd?: number` → error subtype `error_max_budget_usd` | `--max-budget-usd` |
|
|
88
|
+
| Effort | `effort?: 'low'\|'medium'\|'high'\|'xhigh'\|'max' \| number` (also runtime `applyFlagSettings({effortLevel})`) | `--effort` |
|
|
89
|
+
| Model | `model?: string` (alias/full id; runtime `setModel()`) | `--model` |
|
|
90
|
+
| Fallback | `fallbackModel?: string` | `--fallback-model` |
|
|
91
|
+
| Turn cap | `maxTurns?: number` → `error_max_turns` | (SDK/`--max-turns`) |
|
|
92
|
+
| Task budget | `taskBudget?: { total: number }` → sent as API `output_config.task_budget` + beta header `task-budgets-2026-03-13` | - |
|
|
93
|
+
| Thinking | `thinking?: ThinkingConfig` (`maxThinkingTokens` deprecated) | - |
|
|
94
|
+
| Betas | `betas?: SdkBeta[]` - only `'context-1m-2025-08-07'` is valid | `--betas` |
|
|
95
|
+
|
|
96
|
+
Result message carries `total_cost_usd`, `usage` (NonNullableUsage), `modelUsage: Record<string, ModelUsage>`, `num_turns`, `duration_ms/api_ms`, `permission_denials` - richer cost/telemetry than parsing CLI stdout. **Every CLI control Loki uses has a typed SDK option.**
|
|
97
|
+
|
|
98
|
+
## 5. The "outcome" / Managed-Agents primitive vs RARV-C (honest assessment)
|
|
99
|
+
There is **no `outcome` primitive in the Agent SDK.** The closest thing is the SDK's built-in agent loop bounded by `maxTurns` / `maxBudgetUsd` / `taskBudget` / structured-output retries - an *effort/budget-bounded* loop, **not** an iterate-until-a-grader-says-done loop.
|
|
100
|
+
|
|
101
|
+
The grader-driven "iterate until an independent rubric passes" primitive is **Managed Agents' Outcomes** (`user.define_outcome` + rubric + `max_iterations`, with `span.outcome_evaluation_*` events and a separate grader model) - a **different product** (hosted REST API, `client.beta.sessions.*` on `@anthropic-ai/sdk`), not the Agent SDK. Per the docs' own comparison table, Managed Agents runs the loop + sandbox on Anthropic infra; the Agent SDK runs in your process.
|
|
102
|
+
|
|
103
|
+
**Honest verdict:** RARV-C already *is* Loki's home-grown Outcomes equivalent (iterate → grade via council → close). Neither the Agent SDK nor Managed Agents Outcomes maps onto it 1:1. Do **not** fabricate a wiring where the SDK "provides" RARV-C - the SDK provides a bounded tool-use loop; the grader/council closure stays Loki's. Managed Agents Outcomes is the nearest hosted analog but would mean re-platforming completion onto Anthropic's grader (out of scope, and would cede Loki's council).
|
|
104
|
+
|
|
105
|
+
## 6. Runtime / auth / cost (verified)
|
|
106
|
+
- **Runtime:** Node, Bun, or Deno - `executable?: 'bun'|'deno'|'node'`. TS SDK bundles the native binary per platform (no separate `claude` install). Python variant (`claude-agent-sdk`) needs Python ≥3.10.
|
|
107
|
+
- **Auth:** `ANTHROPIC_API_KEY` (docs). Also Bedrock (`CLAUDE_CODE_USE_BEDROCK=1`), Claude Platform on AWS (`CLAUDE_CODE_USE_ANTHROPIC_AWS=1` + `ANTHROPIC_AWS_WORKSPACE_ID`), Vertex (`CLAUDE_CODE_USE_VERTEX=1`), Foundry (`CLAUDE_CODE_USE_FOUNDRY=1`). **Anthropic explicitly prohibits third-party products from using claude.ai login / subscription rate limits via the Agent SDK - API-key auth only.** No interactive login (a real improvement over the CLI's OAuth `claude auth`).
|
|
108
|
+
- **Cost:** standard per-token API billing (surfaced live as `total_cost_usd` on the result). Same underlying `/v1/messages` cost as the CLI - the CLI's cost is API cost; the SDK doesn't change the cost model, it just reports it structurally.
|
|
109
|
+
|
|
110
|
+
## 7. Honest gaps (what each does that the other doesn't)
|
|
111
|
+
|
|
112
|
+
**Agent SDK gains over Loki's `claude -p` wrapper:**
|
|
113
|
+
- Typed streaming messages (`SDKMessage` union) instead of stdout parsing; structured `SDKResultSuccess`/`SDKResultError` with cost/usage/`structured_output`.
|
|
114
|
+
- In-process custom tools (`tool()`), in-process hooks, programmatic subagents (`agents`), programmatic MCP.
|
|
115
|
+
- Mid-session control (`interrupt`, `setModel`, `setPermissionMode`, `applyFlagSettings`, `streamInput`, `rewindFiles`) - no equivalent when you shell out to `claude -p`.
|
|
116
|
+
- `startup()`/`WarmQuery` for low-latency warm spares.
|
|
117
|
+
|
|
118
|
+
**Provider-agnostic caveat (per founder's constraint):** the Agent SDK is Anthropic/Claude-Code-specific. It does NOT give Loki Codex/Cline/Aider. Those routes need their own story (the OpenAI-compatible layer, explicitly out of scope here). Note the Agent SDK *can* target Bedrock/Vertex/Foundry via env vars, but that's still Claude models.
|
|
119
|
+
|
|
120
|
+
**What the Agent SDK does NOT do / gaps vs the premise:**
|
|
121
|
+
- **Does not remove the native binary** (finding above) - it spawns a bundled Claude Code executable per platform. "Library, just an API key, no binary" is only true of `@anthropic-ai/sdk` (Client SDK).
|
|
122
|
+
- No grader/Outcomes primitive (RARV-C stays Loki's; §5).
|
|
123
|
+
- No built-in vector memory (Loki's `memory/` package has no SDK analog; wire via MCP/hooks).
|
|
124
|
+
- Only one valid `SdkBeta` (`context-1m-2025-08-07`) - other API betas (fast-mode, compaction, etc.) aren't first-class SDK options.
|
|
125
|
+
- WebFetch-only claims I could NOT verify in the `.d.ts` and mark **UNVERIFIED**: the exact `SDKAssistantMessage`/`SDKPartialAssistantMessage` content-block shapes, `ThinkingConfig` as a 3-variant union, and `CanUseTool`'s full options object. The `thinking?: ThinkingConfig` field, `includePartialMessages?: boolean`, and `canUseTool?: CanUseTool` fields **are** confirmed present; their inner shapes should be re-read from `sdk.d.ts` before you write code against them.
|
|
126
|
+
|
|
127
|
+
Verified type-def file (kept for the implementer): `/private/tmp/claude-501/-Users-lokesh-git-lokimode-anthropic/8db286db-3006-49f5-9dc6-cb241e559d45/scratchpad/package/sdk.d.ts` (line refs above are from this file, v0.3.207).
|
|
128
|
+
|
|
129
|
+
**Recommendation to fold into the v8 plan:** the migration decision is not "Agent SDK vs bash" - it's a three-way fork (Agent SDK = bundled binary + full harness; Client SDK = pure API, Loki owns the loop; Managed Agents = hosted loop+sandbox+Outcomes). Only the Client SDK path delivers the founder's stated "no binary, API-key-only container" win, at the cost of re-owning the loop Loki mostly already owns. Surface this explicitly before locking scope.
|