@draht/rlm 2026.7.12-1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md ADDED
@@ -0,0 +1,167 @@
1
+ # @draht/rlm
2
+
3
+ Recursive Language Models (RLM) — the inference-time scaffold from Zhang,
4
+ Kraska, Khattab (2026), *Recursive Language Models* (arXiv:2512.24601):
5
+ root-LLM-produces-code → REPL-executes → truncated-stdout → history-append →
6
+ FINAL-check. Lets a root model answer questions about inputs far larger than
7
+ its own context window by writing Python that inspects, chunks, and searches
8
+ the content from inside a sandboxed REPL, optionally recursing into
9
+ sub-`llm_query` calls, instead of the caller ever loading the raw content
10
+ into a single prompt.
11
+
12
+ ## When to prefer RLM over a normal agent read
13
+
14
+ Reach for RLM when the input genuinely doesn't fit the model's context
15
+ window, or fits only by truncating away content that might matter:
16
+
17
+ - A log file, dataset dump, or exported knowledge base that's tens or
18
+ hundreds of megabytes — orders of magnitude past any model's window.
19
+ - A directory/glob of source files too large to read whole without evicting
20
+ earlier conversation context.
21
+ - Any case where you'd otherwise have to guess which prefix/suffix/sample of
22
+ the input to truncate to, and a wrong guess means missing the answer.
23
+
24
+ Don't reach for RLM when the input already fits comfortably in the model's
25
+ context window — a normal read is faster, cheaper, and easier to debug. RLM
26
+ trades one direct read for several extra model round trips (see below); that
27
+ trade only pays off when the alternative is truncation-induced information
28
+ loss, not just "this file is somewhat long."
29
+
30
+ ## Cost envelope
31
+
32
+ A normal read is one call: the whole input goes into the prompt, the model
33
+ answers once. RLM is **not** universally cheaper than that — it's a
34
+ different trade:
35
+
36
+ - The root model makes one call per REPL step (a handful to a few dozen
37
+ steps for a typical run), and each step's Python may itself call
38
+ `llm_query(...)`, which is a *second* model call (typically to a smaller
39
+ "rlm-sub" role — see `router-session.ts`'s `RlmRole`) that also costs
40
+ money.
41
+ - So RLM's total cost is `sum(root-step costs) + sum(sub-call costs)` versus
42
+ a baseline's single call. RLM wins when the baseline's single call would
43
+ otherwise require either failing outright (input doesn't fit at all) or
44
+ paying for a much larger context window / re-reading the input multiple
45
+ times across a conversation to compensate for truncation — not in the
46
+ general case.
47
+ - **Don't guess — measure.** `src/cost-comparison.ts`'s `compareCost()`
48
+ computes both numbers for one trajectory: `rlmCostUsd` from the trajectory's
49
+ own logged cost entries (`readCostLog`/`getSessionCosts` from
50
+ `@draht/router`, filtered by `trajectoryId`), and `baselineCostUsd` as the
51
+ estimated cost (via `@draht/router`'s `estimateCost`) of a single call that
52
+ truncates the same content down to the target model's context window
53
+ (first-N-characters-that-fit — a simple, named strategy, not a claim that
54
+ it's the smartest possible truncation). Run it against a real trajectory ID
55
+ before assuming RLM is the cheaper (or more expensive) option for your
56
+ specific input/model pair; `.draht/rlm/cost-comparison-report.json` is
57
+ where a comparison run writes its report.
58
+
59
+ ## Observability: trajectory logs & replay
60
+
61
+ Every `createRouterBackedSession` run writes a newline-delimited JSON
62
+ trajectory log to `.draht/rlm/<trajectoryId>.jsonl` (`src/trajectory.ts`) —
63
+ one `TrajectoryStepEntry` per REPL step (its code, truncated stdout, error if
64
+ any, the sub-calls it triggered, and that step's own cost) plus one terminal
65
+ `TrajectoryFinalEntry` once the session resolves. `trajectoryId` is the same
66
+ ID already used to tag the session's `@draht/router` cost-log entries, so a
67
+ trajectory's logs and its cost accounting always correlate by the same key.
68
+
69
+ Because the whole run is on disk, it can be reconstructed with **zero LLM
70
+ calls**:
71
+
72
+ ```bash
73
+ draht rlm replay <trajectory-id>
74
+ ```
75
+
76
+ reads the JSONL log via `readTrajectory()` and prints the final answer (add a
77
+ verbose flag for the full step-by-step trace) — no router, no model, no
78
+ network access on that code path at all. Useful for auditing what a session
79
+ actually did, debugging a run that didn't reach `FINAL`, or reviewing cost
80
+ without re-running anything.
81
+
82
+ The `test/s-niah.test.ts` regression suite exercises recall at 10x and 100x a
83
+ mocked root model's context window (synthetic needle-in-haystack at ~4M and
84
+ ~40M characters, against a scripted mock `rootLlm` — no real model calls) as
85
+ a standing check that the scaffold still finds a planted needle at scale.
86
+
87
+ ## Worked example
88
+
89
+ Say `deploy.log` is a 40 MB deployment log and you need to know what caused
90
+ an outage buried somewhere in it — too large to paste into a normal prompt.
91
+ The same question, three ways:
92
+
93
+ **Interactive `/rlm` command** (inside a `draht` session):
94
+
95
+ ```
96
+ /rlm deploy.log What caused the outage around the 03:00 UTC deploy?
97
+ ```
98
+
99
+ **`draht rlm` CLI** (scriptable, one-shot):
100
+
101
+ ```bash
102
+ draht rlm --input ./deploy.log --query "What caused the outage around the 03:00 UTC deploy?"
103
+ ```
104
+
105
+ **`rlm_query` tool** (an agent defers the read itself, mid-session, instead
106
+ of loading the log into its own context):
107
+
108
+ ```json
109
+ {
110
+ "tool": "rlm_query",
111
+ "input": {
112
+ "input": "deploy.log",
113
+ "query": "What caused the outage around the 03:00 UTC deploy?"
114
+ }
115
+ }
116
+ ```
117
+
118
+ All three share the same path under the hood: `parseInputArg`/`loadInput`
119
+ resolve `deploy.log` into content, `createRouterBackedSession` wraps it as
120
+ the REPL's `context` with the question prepended as a header, and the root
121
+ model writes Python to grep/chunk/search `context` — recursing into
122
+ `llm_query` sub-calls for individual chunks as needed — until it calls
123
+ `FINAL(...)`/`FINAL_VAR(...)` with the answer. `--input`/`input` also accepts
124
+ a glob (`"src/**/*.ts"`), an `http(s)://` URL, or `knowledge:<client-slug>`
125
+ for a named client knowledge base.
126
+
127
+ To check whether that particular run was worth it cost-wise, take the
128
+ `trajectoryId` of its `.draht/rlm/<trajectoryId>.jsonl` log (or the matching
129
+ `trajectoryId` field in `.draht/cost-log.jsonl`) and feed it into
130
+ `compareCost()` (see Cost envelope, above) rather than assuming either
131
+ direction.
132
+
133
+ ## Development
134
+
135
+ ```bash
136
+ npm install # from repo root, resolves this workspace package
137
+ cd packages/rlm
138
+ npx tsc --noEmit # typecheck
139
+ npm test # vitest
140
+ ```
141
+
142
+ ## Package layout
143
+
144
+ - `src/session.ts` — `RlmSession` root loop (step/run, `FINAL`/`FINAL_VAR`
145
+ sentinels, constant-size history).
146
+ - `src/router-session.ts` — `createRouterBackedSession`, the production
147
+ `@draht/router`-backed factory (prompt-tier selection, cost logging,
148
+ trajectory logging).
149
+ - `src/trajectory.ts` — trajectory JSONL append/read (this phase).
150
+ - `src/cost-comparison.ts` — RLM-vs-baseline cost harness (this phase).
151
+ - `src/loaders.ts` — `--input`/`/rlm <input>` source loaders (file, glob,
152
+ URL, `knowledge:<slug>`).
153
+ - `src/prompts.ts` — tiered system-prompt templates, selected by the root
154
+ model's context window.
155
+ - `python/repl_driver.py` — the sandboxed Python REPL driver.
156
+ - `packages/coding-agent/src/rlm-cli.ts` — `draht rlm` CLI subcommand.
157
+ - `packages/rlm-agent/` — `/rlm` command + `rlm_query` tool extension for
158
+ normal agent sessions.
159
+
160
+ ## AGENTS.md
161
+
162
+ RLM is a cross-cutting coding-agent capability, not a property of any single
163
+ generated project's stack — none of `packages/templates/src/*.md`'s
164
+ per-stack templates (Astro, Go/gRPC, SST TypeScript) is the right home for a
165
+ "when to use RLM" note, and this repo's own root `AGENTS.md` is a
166
+ contributor-workflow doc, not a feature reference. This README is the
167
+ primary documentation surface for RLM; no template changes were made.
@@ -0,0 +1,81 @@
1
+ /**
2
+ * Cost-comparison harness: RLM trajectory cost vs. a truncate-and-single-call
3
+ * baseline for the same task.
4
+ *
5
+ * `rlmCostUsd` is the real, already-incurred cost of a trajectory recovered
6
+ * from `@draht/router`'s cost log (every root + sub call `createRouterBackedSession`
7
+ * logs is tagged with one `trajectoryId` -- see router-session.ts). There is
8
+ * nothing to "compare" about that half; it's just a sum-and-filter over
9
+ * already-logged `CostEntry` rows.
10
+ *
11
+ * `baselineCostUsd` is necessarily an *estimate*, not a measurement: it models
12
+ * the cost of the simplest plausible non-RLM alternative for an oversize
13
+ * task -- truncate the context down to whatever fits the target model's
14
+ * context window, then make one single call -- and prices that hypothetical
15
+ * call via `@draht/router`'s `estimateCost`. The specific truncation strategy
16
+ * modeled here is deliberately the crudest one: keep the first N characters
17
+ * of the context that fit the window (a `CHARS_PER_TOKEN` heuristic converts
18
+ * the token-denominated `contextWindow` into a character budget). A smarter
19
+ * baseline (e.g. picking the *most relevant* N characters via embedding
20
+ * search, or summarizing before truncating) would likely produce different
21
+ * numbers -- that's out of scope here. This harness answers "what would the
22
+ * cheapest naive single-call alternative have cost", not "what is the best
23
+ * possible non-RLM approach". See
24
+ * .planning/phases/30-eval-observability-docs/30-01-PLAN.md, Architecture
25
+ * section 3, and its Risks section.
26
+ */
27
+ /** Which target model the truncate-and-single-call baseline is priced against. */
28
+ export interface BaselineModel {
29
+ provider: string;
30
+ modelName: string;
31
+ /** Context window, in tokens -- used to compute how much of the context the baseline call could fit after truncation. */
32
+ contextWindow: number;
33
+ }
34
+ /** Result of comparing one RLM trajectory's real cost against a truncate-and-single-call baseline for the same task. */
35
+ export interface CostComparisonResult {
36
+ /**
37
+ * Identifies which task/trajectory this comparison covers. There is no
38
+ * separate task-name concept tracked elsewhere in this codebase, so this
39
+ * is the `trajectoryId` itself -- callers that want a human-readable label
40
+ * should track that mapping themselves.
41
+ */
42
+ task: string;
43
+ /** Sum of every cost-log entry (root call + every `llm_query` sub-call) tagged with this trajectoryId. */
44
+ rlmCostUsd: number;
45
+ /**
46
+ * Estimated cost of truncating the context to fit `model`'s window and
47
+ * making one single call instead (see the module doc for the exact
48
+ * truncation approach modeled).
49
+ */
50
+ baselineCostUsd: number;
51
+ /** Number of root-LLM steps the RLM trajectory took (one "rlm-root"-role cost entry per step -- see router-session.ts's `rootLlm`). */
52
+ rlmSteps: number;
53
+ /** The full (untruncated) context length, in characters, the RLM trajectory operated over. */
54
+ contextLengthChars: number;
55
+ }
56
+ /**
57
+ * Compares an already-run RLM trajectory's real cost (read back from the
58
+ * cost log at `costLogPath`, filtered by `trajectoryId`) against the
59
+ * estimated cost of a truncate-and-single-call baseline for the same task:
60
+ * truncate `contextLengthChars` down to whatever fits `model.contextWindow`
61
+ * (first-N-characters, converted via `CHARS_PER_TOKEN`), then price one call
62
+ * to `model` via `@draht/router`'s `estimateCost`.
63
+ *
64
+ * Doesn't run anything itself -- the RLM trajectory must already have run
65
+ * (via `createRouterBackedSession`, logging its cost entries under
66
+ * `costLogPath`) before calling this; the baseline call is hypothetical and
67
+ * never actually dispatched. This function is pure arithmetic over its
68
+ * inputs plus the cost log, and makes no claim about which side is cheaper --
69
+ * that depends entirely on the task.
70
+ */
71
+ export declare function compareCost(trajectoryId: string, contextLengthChars: number, model: BaselineModel, costLogPath?: string): CostComparisonResult;
72
+ /**
73
+ * Writes a `CostComparisonResult` to disk as the "eval report" R30-EVAL.3
74
+ * asks for, creating parent directories as needed. Defaults to
75
+ * `.draht/rlm/cost-comparison-report.json` (relative to `process.cwd()`),
76
+ * mirroring `trajectory.ts`'s `DEFAULT_LOG_DIR` convention -- callers that
77
+ * don't want to touch that real location (e.g. tests) should pass an
78
+ * explicit `reportPath` instead.
79
+ */
80
+ export declare function writeCostComparisonReport(result: CostComparisonResult, reportPath?: string): void;
81
+ //# sourceMappingURL=cost-comparison.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"cost-comparison.d.ts","sourceRoot":"","sources":["../src/cost-comparison.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;GAyBG;AAMH,kFAAkF;AAClF,MAAM,WAAW,aAAa;IAC7B,QAAQ,EAAE,MAAM,CAAC;IACjB,SAAS,EAAE,MAAM,CAAC;IAClB,yHAAyH;IACzH,aAAa,EAAE,MAAM,CAAC;CACtB;AAED,wHAAwH;AACxH,MAAM,WAAW,oBAAoB;IACpC;;;;;OAKG;IACH,IAAI,EAAE,MAAM,CAAC;IACb,0GAA0G;IAC1G,UAAU,EAAE,MAAM,CAAC;IACnB;;;;OAIG;IACH,eAAe,EAAE,MAAM,CAAC;IACxB,uIAAuI;IACvI,QAAQ,EAAE,MAAM,CAAC;IACjB,8FAA8F;IAC9F,kBAAkB,EAAE,MAAM,CAAC;CAC3B;AAuBD;;;;;;;;;;;;;;GAcG;AACH,wBAAgB,WAAW,CAC1B,YAAY,EAAE,MAAM,EACpB,kBAAkB,EAAE,MAAM,EAC1B,KAAK,EAAE,aAAa,EACpB,WAAW,CAAC,EAAE,MAAM,GAClB,oBAAoB,CAwBtB;AAED;;;;;;;GAOG;AACH,wBAAgB,yBAAyB,CAAC,MAAM,EAAE,oBAAoB,EAAE,UAAU,CAAC,EAAE,MAAM,GAAG,IAAI,CAIjG"}
@@ -0,0 +1,93 @@
1
+ /**
2
+ * Cost-comparison harness: RLM trajectory cost vs. a truncate-and-single-call
3
+ * baseline for the same task.
4
+ *
5
+ * `rlmCostUsd` is the real, already-incurred cost of a trajectory recovered
6
+ * from `@draht/router`'s cost log (every root + sub call `createRouterBackedSession`
7
+ * logs is tagged with one `trajectoryId` -- see router-session.ts). There is
8
+ * nothing to "compare" about that half; it's just a sum-and-filter over
9
+ * already-logged `CostEntry` rows.
10
+ *
11
+ * `baselineCostUsd` is necessarily an *estimate*, not a measurement: it models
12
+ * the cost of the simplest plausible non-RLM alternative for an oversize
13
+ * task -- truncate the context down to whatever fits the target model's
14
+ * context window, then make one single call -- and prices that hypothetical
15
+ * call via `@draht/router`'s `estimateCost`. The specific truncation strategy
16
+ * modeled here is deliberately the crudest one: keep the first N characters
17
+ * of the context that fit the window (a `CHARS_PER_TOKEN` heuristic converts
18
+ * the token-denominated `contextWindow` into a character budget). A smarter
19
+ * baseline (e.g. picking the *most relevant* N characters via embedding
20
+ * search, or summarizing before truncating) would likely produce different
21
+ * numbers -- that's out of scope here. This harness answers "what would the
22
+ * cheapest naive single-call alternative have cost", not "what is the best
23
+ * possible non-RLM approach". See
24
+ * .planning/phases/30-eval-observability-docs/30-01-PLAN.md, Architecture
25
+ * section 3, and its Risks section.
26
+ */
27
+ import { mkdirSync, writeFileSync } from "node:fs";
28
+ import { dirname, resolve } from "node:path";
29
+ import { estimateCost, readCostLog } from "@draht/router";
30
+ /**
31
+ * Rough heuristic: ~4 characters per token for English-ish text. This is the
32
+ * same order-of-magnitude approximation used broadly across LLM tooling when
33
+ * an exact tokenizer isn't wired in for a given model -- it is NOT
34
+ * model-exact, and the `baselineCostUsd` this produces should be read as
35
+ * "same ballpark", not "precise". See the module doc above and the Risks
36
+ * section of the phase 30 plan.
37
+ */
38
+ const CHARS_PER_TOKEN = 4;
39
+ /**
40
+ * Fixed assumed output length (tokens) for the baseline's single call -- a
41
+ * plausible "here's the answer" response size. Deliberately not modeled
42
+ * per-task: this harness needs a reasonable, explicitly-documented constant,
43
+ * not a task-specific prediction of how long an answer would be.
44
+ */
45
+ const BASELINE_OUTPUT_TOKENS = 500;
46
+ /** Default path the "eval report" JSON is written to/read from, relative to `process.cwd()`. */
47
+ const DEFAULT_REPORT_PATH = ".draht/rlm/cost-comparison-report.json";
48
+ /**
49
+ * Compares an already-run RLM trajectory's real cost (read back from the
50
+ * cost log at `costLogPath`, filtered by `trajectoryId`) against the
51
+ * estimated cost of a truncate-and-single-call baseline for the same task:
52
+ * truncate `contextLengthChars` down to whatever fits `model.contextWindow`
53
+ * (first-N-characters, converted via `CHARS_PER_TOKEN`), then price one call
54
+ * to `model` via `@draht/router`'s `estimateCost`.
55
+ *
56
+ * Doesn't run anything itself -- the RLM trajectory must already have run
57
+ * (via `createRouterBackedSession`, logging its cost entries under
58
+ * `costLogPath`) before calling this; the baseline call is hypothetical and
59
+ * never actually dispatched. This function is pure arithmetic over its
60
+ * inputs plus the cost log, and makes no claim about which side is cheaper --
61
+ * that depends entirely on the task.
62
+ */
63
+ export function compareCost(trajectoryId, contextLengthChars, model, costLogPath) {
64
+ const entries = readCostLog(costLogPath).filter((entry) => entry.trajectoryId === trajectoryId);
65
+ const rlmCostUsd = entries.reduce((sum, entry) => sum + entry.estimatedCostUsd, 0);
66
+ // Exactly one "rlm-root"-role cost entry is logged per root-LLM step (see
67
+ // router-session.ts's `logRouterCall("rlm-root", ...)` inside `rootLlm`) --
68
+ // sub-calls (role "rlm-sub") don't count as separate steps.
69
+ const rlmSteps = entries.filter((entry) => entry.role === "rlm-root").length;
70
+ const truncatedInputTokens = Math.min(model.contextWindow, Math.ceil(contextLengthChars / CHARS_PER_TOKEN));
71
+ const baselineCostUsd = estimateCost(model.provider, model.modelName, truncatedInputTokens, BASELINE_OUTPUT_TOKENS, 0);
72
+ return {
73
+ task: trajectoryId,
74
+ rlmCostUsd,
75
+ baselineCostUsd,
76
+ rlmSteps,
77
+ contextLengthChars,
78
+ };
79
+ }
80
+ /**
81
+ * Writes a `CostComparisonResult` to disk as the "eval report" R30-EVAL.3
82
+ * asks for, creating parent directories as needed. Defaults to
83
+ * `.draht/rlm/cost-comparison-report.json` (relative to `process.cwd()`),
84
+ * mirroring `trajectory.ts`'s `DEFAULT_LOG_DIR` convention -- callers that
85
+ * don't want to touch that real location (e.g. tests) should pass an
86
+ * explicit `reportPath` instead.
87
+ */
88
+ export function writeCostComparisonReport(result, reportPath) {
89
+ const path = resolve(reportPath ?? DEFAULT_REPORT_PATH);
90
+ mkdirSync(dirname(path), { recursive: true });
91
+ writeFileSync(path, `${JSON.stringify(result, null, 2)}\n`, "utf-8");
92
+ }
93
+ //# sourceMappingURL=cost-comparison.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"cost-comparison.js","sourceRoot":"","sources":["../src/cost-comparison.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;GAyBG;AAEH,OAAO,EAAE,SAAS,EAAE,aAAa,EAAE,MAAM,SAAS,CAAC;AACnD,OAAO,EAAE,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AAC7C,OAAO,EAAE,YAAY,EAAE,WAAW,EAAE,MAAM,eAAe,CAAC;AAiC1D;;;;;;;GAOG;AACH,MAAM,eAAe,GAAG,CAAC,CAAC;AAE1B;;;;;GAKG;AACH,MAAM,sBAAsB,GAAG,GAAG,CAAC;AAEnC,gGAAgG;AAChG,MAAM,mBAAmB,GAAG,wCAAwC,CAAC;AAErE;;;;;;;;;;;;;;GAcG;AACH,MAAM,UAAU,WAAW,CAC1B,YAAoB,EACpB,kBAA0B,EAC1B,KAAoB,EACpB,WAAoB;IAEpB,MAAM,OAAO,GAAG,WAAW,CAAC,WAAW,CAAC,CAAC,MAAM,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,YAAY,KAAK,YAAY,CAAC,CAAC;IAChG,MAAM,UAAU,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,KAAK,EAAE,EAAE,CAAC,GAAG,GAAG,KAAK,CAAC,gBAAgB,EAAE,CAAC,CAAC,CAAC;IACnF,0EAA0E;IAC1E,4EAA4E;IAC5E,4DAA4D;IAC5D,MAAM,QAAQ,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,IAAI,KAAK,UAAU,CAAC,CAAC,MAAM,CAAC;IAE7E,MAAM,oBAAoB,GAAG,IAAI,CAAC,GAAG,CAAC,KAAK,CAAC,aAAa,EAAE,IAAI,CAAC,IAAI,CAAC,kBAAkB,GAAG,eAAe,CAAC,CAAC,CAAC;IAC5G,MAAM,eAAe,GAAG,YAAY,CACnC,KAAK,CAAC,QAAQ,EACd,KAAK,CAAC,SAAS,EACf,oBAAoB,EACpB,sBAAsB,EACtB,CAAC,CACD,CAAC;IAEF,OAAO;QACN,IAAI,EAAE,YAAY;QAClB,UAAU;QACV,eAAe;QACf,QAAQ;QACR,kBAAkB;KAClB,CAAC;AACH,CAAC;AAED;;;;;;;GAOG;AACH,MAAM,UAAU,yBAAyB,CAAC,MAA4B,EAAE,UAAmB;IAC1F,MAAM,IAAI,GAAG,OAAO,CAAC,UAAU,IAAI,mBAAmB,CAAC,CAAC;IACxD,SAAS,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;IAC9C,aAAa,CAAC,IAAI,EAAE,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC,IAAI,EAAE,OAAO,CAAC,CAAC;AACtE,CAAC","sourcesContent":["/**\n * Cost-comparison harness: RLM trajectory cost vs. a truncate-and-single-call\n * baseline for the same task.\n *\n * `rlmCostUsd` is the real, already-incurred cost of a trajectory recovered\n * from `@draht/router`'s cost log (every root + sub call `createRouterBackedSession`\n * logs is tagged with one `trajectoryId` -- see router-session.ts). There is\n * nothing to \"compare\" about that half; it's just a sum-and-filter over\n * already-logged `CostEntry` rows.\n *\n * `baselineCostUsd` is necessarily an *estimate*, not a measurement: it models\n * the cost of the simplest plausible non-RLM alternative for an oversize\n * task -- truncate the context down to whatever fits the target model's\n * context window, then make one single call -- and prices that hypothetical\n * call via `@draht/router`'s `estimateCost`. The specific truncation strategy\n * modeled here is deliberately the crudest one: keep the first N characters\n * of the context that fit the window (a `CHARS_PER_TOKEN` heuristic converts\n * the token-denominated `contextWindow` into a character budget). A smarter\n * baseline (e.g. picking the *most relevant* N characters via embedding\n * search, or summarizing before truncating) would likely produce different\n * numbers -- that's out of scope here. This harness answers \"what would the\n * cheapest naive single-call alternative have cost\", not \"what is the best\n * possible non-RLM approach\". See\n * .planning/phases/30-eval-observability-docs/30-01-PLAN.md, Architecture\n * section 3, and its Risks section.\n */\n\nimport { mkdirSync, writeFileSync } from \"node:fs\";\nimport { dirname, resolve } from \"node:path\";\nimport { estimateCost, readCostLog } from \"@draht/router\";\n\n/** Which target model the truncate-and-single-call baseline is priced against. */\nexport interface BaselineModel {\n\tprovider: string;\n\tmodelName: string;\n\t/** Context window, in tokens -- used to compute how much of the context the baseline call could fit after truncation. */\n\tcontextWindow: number;\n}\n\n/** Result of comparing one RLM trajectory's real cost against a truncate-and-single-call baseline for the same task. */\nexport interface CostComparisonResult {\n\t/**\n\t * Identifies which task/trajectory this comparison covers. There is no\n\t * separate task-name concept tracked elsewhere in this codebase, so this\n\t * is the `trajectoryId` itself -- callers that want a human-readable label\n\t * should track that mapping themselves.\n\t */\n\ttask: string;\n\t/** Sum of every cost-log entry (root call + every `llm_query` sub-call) tagged with this trajectoryId. */\n\trlmCostUsd: number;\n\t/**\n\t * Estimated cost of truncating the context to fit `model`'s window and\n\t * making one single call instead (see the module doc for the exact\n\t * truncation approach modeled).\n\t */\n\tbaselineCostUsd: number;\n\t/** Number of root-LLM steps the RLM trajectory took (one \"rlm-root\"-role cost entry per step -- see router-session.ts's `rootLlm`). */\n\trlmSteps: number;\n\t/** The full (untruncated) context length, in characters, the RLM trajectory operated over. */\n\tcontextLengthChars: number;\n}\n\n/**\n * Rough heuristic: ~4 characters per token for English-ish text. This is the\n * same order-of-magnitude approximation used broadly across LLM tooling when\n * an exact tokenizer isn't wired in for a given model -- it is NOT\n * model-exact, and the `baselineCostUsd` this produces should be read as\n * \"same ballpark\", not \"precise\". See the module doc above and the Risks\n * section of the phase 30 plan.\n */\nconst CHARS_PER_TOKEN = 4;\n\n/**\n * Fixed assumed output length (tokens) for the baseline's single call -- a\n * plausible \"here's the answer\" response size. Deliberately not modeled\n * per-task: this harness needs a reasonable, explicitly-documented constant,\n * not a task-specific prediction of how long an answer would be.\n */\nconst BASELINE_OUTPUT_TOKENS = 500;\n\n/** Default path the \"eval report\" JSON is written to/read from, relative to `process.cwd()`. */\nconst DEFAULT_REPORT_PATH = \".draht/rlm/cost-comparison-report.json\";\n\n/**\n * Compares an already-run RLM trajectory's real cost (read back from the\n * cost log at `costLogPath`, filtered by `trajectoryId`) against the\n * estimated cost of a truncate-and-single-call baseline for the same task:\n * truncate `contextLengthChars` down to whatever fits `model.contextWindow`\n * (first-N-characters, converted via `CHARS_PER_TOKEN`), then price one call\n * to `model` via `@draht/router`'s `estimateCost`.\n *\n * Doesn't run anything itself -- the RLM trajectory must already have run\n * (via `createRouterBackedSession`, logging its cost entries under\n * `costLogPath`) before calling this; the baseline call is hypothetical and\n * never actually dispatched. This function is pure arithmetic over its\n * inputs plus the cost log, and makes no claim about which side is cheaper --\n * that depends entirely on the task.\n */\nexport function compareCost(\n\ttrajectoryId: string,\n\tcontextLengthChars: number,\n\tmodel: BaselineModel,\n\tcostLogPath?: string,\n): CostComparisonResult {\n\tconst entries = readCostLog(costLogPath).filter((entry) => entry.trajectoryId === trajectoryId);\n\tconst rlmCostUsd = entries.reduce((sum, entry) => sum + entry.estimatedCostUsd, 0);\n\t// Exactly one \"rlm-root\"-role cost entry is logged per root-LLM step (see\n\t// router-session.ts's `logRouterCall(\"rlm-root\", ...)` inside `rootLlm`) --\n\t// sub-calls (role \"rlm-sub\") don't count as separate steps.\n\tconst rlmSteps = entries.filter((entry) => entry.role === \"rlm-root\").length;\n\n\tconst truncatedInputTokens = Math.min(model.contextWindow, Math.ceil(contextLengthChars / CHARS_PER_TOKEN));\n\tconst baselineCostUsd = estimateCost(\n\t\tmodel.provider,\n\t\tmodel.modelName,\n\t\ttruncatedInputTokens,\n\t\tBASELINE_OUTPUT_TOKENS,\n\t\t0,\n\t);\n\n\treturn {\n\t\ttask: trajectoryId,\n\t\trlmCostUsd,\n\t\tbaselineCostUsd,\n\t\trlmSteps,\n\t\tcontextLengthChars,\n\t};\n}\n\n/**\n * Writes a `CostComparisonResult` to disk as the \"eval report\" R30-EVAL.3\n * asks for, creating parent directories as needed. Defaults to\n * `.draht/rlm/cost-comparison-report.json` (relative to `process.cwd()`),\n * mirroring `trajectory.ts`'s `DEFAULT_LOG_DIR` convention -- callers that\n * don't want to touch that real location (e.g. tests) should pass an\n * explicit `reportPath` instead.\n */\nexport function writeCostComparisonReport(result: CostComparisonResult, reportPath?: string): void {\n\tconst path = resolve(reportPath ?? DEFAULT_REPORT_PATH);\n\tmkdirSync(dirname(path), { recursive: true });\n\twriteFileSync(path, `${JSON.stringify(result, null, 2)}\\n`, \"utf-8\");\n}\n"]}
@@ -0,0 +1,11 @@
1
+ export type { InputSource, LoadedInput } from "./loaders.js";
2
+ export { loadInput, parseInputArg } from "./loaders.js";
3
+ export type { PromptTier, PromptVars } from "./prompts.js";
4
+ export { renderPrompt, selectTier } from "./prompts.js";
5
+ export type { CreateRouterBackedSessionOptions } from "./router-session.js";
6
+ export { createRouterBackedSession } from "./router-session.js";
7
+ export { extractPythonCode, pythonReprToValue, RlmSession, truncateStdout } from "./session.js";
8
+ export type { TrajectoryFinalEntry, TrajectoryStepEntry } from "./trajectory.js";
9
+ export { appendTrajectoryEntry, readTrajectory } from "./trajectory.js";
10
+ export type { RlmHistoryEntry, RlmResult, RlmResultKind, RlmSessionOptions } from "./types.js";
11
+ //# sourceMappingURL=index.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAeA,YAAY,EAAE,WAAW,EAAE,WAAW,EAAE,MAAM,cAAc,CAAC;AAC7D,OAAO,EAAE,SAAS,EAAE,aAAa,EAAE,MAAM,cAAc,CAAC;AACxD,YAAY,EAAE,UAAU,EAAE,UAAU,EAAE,MAAM,cAAc,CAAC;AAC3D,OAAO,EAAE,YAAY,EAAE,UAAU,EAAE,MAAM,cAAc,CAAC;AACxD,YAAY,EAAE,gCAAgC,EAAE,MAAM,qBAAqB,CAAC;AAC5E,OAAO,EAAE,yBAAyB,EAAE,MAAM,qBAAqB,CAAC;AAChE,OAAO,EAAE,iBAAiB,EAAE,iBAAiB,EAAE,UAAU,EAAE,cAAc,EAAE,MAAM,cAAc,CAAC;AAMhG,YAAY,EAAE,oBAAoB,EAAE,mBAAmB,EAAE,MAAM,iBAAiB,CAAC;AACjF,OAAO,EAAE,qBAAqB,EAAE,cAAc,EAAE,MAAM,iBAAiB,CAAC;AACxE,YAAY,EAAE,eAAe,EAAE,SAAS,EAAE,aAAa,EAAE,iBAAiB,EAAE,MAAM,YAAY,CAAC"}
package/dist/index.js ADDED
@@ -0,0 +1,14 @@
1
+ // @draht/rlm — Recursive Language Models core primitives.
2
+ //
3
+ // The root loop (`RlmSession`) and its Python REPL driver
4
+ // (`python/repl_driver.py`) implement the inference-time scaffold from
5
+ // Zhang, Kraska, Khattab (2026), *Recursive Language Models*
6
+ // (arXiv:2512.24601). Real `@draht/router` wiring for `llmQuery`/the root
7
+ // LLM, sandboxing, and CLI integration land in later phases (27-29); see
8
+ // .planning/phases/26-rlm-core-primitives/26-01-PLAN.md.
9
+ export { loadInput, parseInputArg } from "./loaders.js";
10
+ export { renderPrompt, selectTier } from "./prompts.js";
11
+ export { createRouterBackedSession } from "./router-session.js";
12
+ export { extractPythonCode, pythonReprToValue, RlmSession, truncateStdout } from "./session.js";
13
+ export { appendTrajectoryEntry, readTrajectory } from "./trajectory.js";
14
+ //# sourceMappingURL=index.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,0DAA0D;AAC1D,EAAE;AACF,0DAA0D;AAC1D,uEAAuE;AACvE,6DAA6D;AAC7D,0EAA0E;AAC1E,yEAAyE;AACzE,yDAAyD;AASzD,OAAO,EAAE,SAAS,EAAE,aAAa,EAAE,MAAM,cAAc,CAAC;AAExD,OAAO,EAAE,YAAY,EAAE,UAAU,EAAE,MAAM,cAAc,CAAC;AAExD,OAAO,EAAE,yBAAyB,EAAE,MAAM,qBAAqB,CAAC;AAChE,OAAO,EAAE,iBAAiB,EAAE,iBAAiB,EAAE,UAAU,EAAE,cAAc,EAAE,MAAM,cAAc,CAAC;AAOhG,OAAO,EAAE,qBAAqB,EAAE,cAAc,EAAE,MAAM,iBAAiB,CAAC","sourcesContent":["// @draht/rlm — Recursive Language Models core primitives.\n//\n// The root loop (`RlmSession`) and its Python REPL driver\n// (`python/repl_driver.py`) implement the inference-time scaffold from\n// Zhang, Kraska, Khattab (2026), *Recursive Language Models*\n// (arXiv:2512.24601). Real `@draht/router` wiring for `llmQuery`/the root\n// LLM, sandboxing, and CLI integration land in later phases (27-29); see\n// .planning/phases/26-rlm-core-primitives/26-01-PLAN.md.\n\n// Input loaders -- turn a raw `--input`/`/rlm <input>` argument (file, glob,\n// URL, or `knowledge:<client-slug>`) into loaded prompt content. See\n// .planning/phases/29-agent-cli-integration/29-01-PLAN.md, Architecture\n// section 1. Re-exported from the package root so consumers outside this\n// package (e.g. `@draht/rlm-agent`, the `draht rlm` CLI) can import them via\n// the public `@draht/rlm` entry point instead of reaching into `src/`.\nexport type { InputSource, LoadedInput } from \"./loaders.js\";\nexport { loadInput, parseInputArg } from \"./loaders.js\";\nexport type { PromptTier, PromptVars } from \"./prompts.js\";\nexport { renderPrompt, selectTier } from \"./prompts.js\";\nexport type { CreateRouterBackedSessionOptions } from \"./router-session.js\";\nexport { createRouterBackedSession } from \"./router-session.js\";\nexport { extractPythonCode, pythonReprToValue, RlmSession, truncateStdout } from \"./session.js\";\n// Trajectory JSONL logging/replay (Phase 30) -- see\n// .planning/phases/30-eval-observability-docs/30-01-PLAN.md, Architecture\n// section 1. Re-exported from the package root so consumers outside this\n// package (e.g. `draht rlm replay`) can import via the public `@draht/rlm`\n// entry point instead of reaching into `src/`.\nexport type { TrajectoryFinalEntry, TrajectoryStepEntry } from \"./trajectory.js\";\nexport { appendTrajectoryEntry, readTrajectory } from \"./trajectory.js\";\nexport type { RlmHistoryEntry, RlmResult, RlmResultKind, RlmSessionOptions } from \"./types.js\";\n"]}
@@ -0,0 +1,44 @@
1
+ /**
2
+ * Input loaders -- turn a raw `--input`/`/rlm <input>` argument into loaded
3
+ * text content ready to become an `RlmSession`'s `prompt`. Covers four kinds
4
+ * of source: a single file, a glob pattern, an `http(s)://` URL, and a named
5
+ * client knowledge base (via `@draht/knowledge`). See
6
+ * .planning/phases/29-agent-cli-integration/29-01-PLAN.md, Architecture
7
+ * section 1.
8
+ *
9
+ * `parseInputArg` classifies the raw string; `loadInput` does the actual
10
+ * I/O. Kept as two functions so callers (CLI, `/rlm` command, `rlm_query`
11
+ * tool) can validate/echo back what kind of source they resolved before
12
+ * paying the cost of loading it.
13
+ */
14
+ export type InputSource = {
15
+ type: "file";
16
+ path: string;
17
+ } | {
18
+ type: "glob";
19
+ pattern: string;
20
+ cwd: string;
21
+ } | {
22
+ type: "url";
23
+ url: string;
24
+ } | {
25
+ type: "knowledge";
26
+ clientSlug: string;
27
+ };
28
+ export interface LoadedInput {
29
+ /** The loaded/concatenated text content. */
30
+ content: string;
31
+ /** What kind of content `content` holds (e.g. "text", "code", "logs", "json") -- see `PromptVars.contextType`. */
32
+ contextType: string;
33
+ }
34
+ /**
35
+ * Classifies a raw `--input`/`/rlm` argument into an {@link InputSource}:
36
+ * - `knowledge:<client-slug>` -> `{ type: "knowledge", clientSlug }`.
37
+ * - `http://...` / `https://...` -> `{ type: "url", url }`.
38
+ * - anything containing glob metacharacters (`* ? { } [ ] !`) -> `{ type: "glob", pattern, cwd }`.
39
+ * - everything else -> `{ type: "file", path }`, resolved against `cwd` if relative.
40
+ */
41
+ export declare function parseInputArg(input: string, cwd: string): InputSource;
42
+ /** Loads the given {@link InputSource}, returning its content and the context type to advertise to the RLM root loop. */
43
+ export declare function loadInput(source: InputSource): Promise<LoadedInput>;
44
+ //# sourceMappingURL=loaders.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"loaders.d.ts","sourceRoot":"","sources":["../src/loaders.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;GAYG;AAMH,MAAM,MAAM,WAAW,GACpB;IAAE,IAAI,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GAC9B;IAAE,IAAI,EAAE,MAAM,CAAC;IAAC,OAAO,EAAE,MAAM,CAAC;IAAC,GAAG,EAAE,MAAM,CAAA;CAAE,GAC9C;IAAE,IAAI,EAAE,KAAK,CAAC;IAAC,GAAG,EAAE,MAAM,CAAA;CAAE,GAC5B;IAAE,IAAI,EAAE,WAAW,CAAC;IAAC,UAAU,EAAE,MAAM,CAAA;CAAE,CAAC;AAE7C,MAAM,WAAW,WAAW;IAC3B,4CAA4C;IAC5C,OAAO,EAAE,MAAM,CAAC;IAChB,kHAAkH;IAClH,WAAW,EAAE,MAAM,CAAC;CACpB;AAkCD;;;;;;GAMG;AACH,wBAAgB,aAAa,CAAC,KAAK,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,GAAG,WAAW,CAiBrE;AAED,yHAAyH;AACzH,wBAAsB,SAAS,CAAC,MAAM,EAAE,WAAW,GAAG,OAAO,CAAC,WAAW,CAAC,CAgBzE"}
@@ -0,0 +1,156 @@
1
+ /**
2
+ * Input loaders -- turn a raw `--input`/`/rlm <input>` argument into loaded
3
+ * text content ready to become an `RlmSession`'s `prompt`. Covers four kinds
4
+ * of source: a single file, a glob pattern, an `http(s)://` URL, and a named
5
+ * client knowledge base (via `@draht/knowledge`). See
6
+ * .planning/phases/29-agent-cli-integration/29-01-PLAN.md, Architecture
7
+ * section 1.
8
+ *
9
+ * `parseInputArg` classifies the raw string; `loadInput` does the actual
10
+ * I/O. Kept as two functions so callers (CLI, `/rlm` command, `rlm_query`
11
+ * tool) can validate/echo back what kind of source they resolved before
12
+ * paying the cost of loading it.
13
+ */
14
+ import { glob, readFile } from "node:fs/promises";
15
+ import * as path from "node:path";
16
+ /**
17
+ * Recognized prefix for the knowledge-base input syntax: `knowledge:<client-slug>`
18
+ * (e.g. `knowledge:acme-corp`), where `<client-slug>` matches `ClientConfig.slug`
19
+ * from `@draht/knowledge`'s `client-config.ts`.
20
+ */
21
+ const KNOWLEDGE_PREFIX = "knowledge:";
22
+ /** Characters that mark an `--input` string as a glob pattern rather than a literal file path. */
23
+ const GLOB_CHARS = /[*?{}[\]!]/;
24
+ const CODE_EXTENSIONS = new Set([
25
+ ".ts",
26
+ ".tsx",
27
+ ".js",
28
+ ".jsx",
29
+ ".mjs",
30
+ ".cjs",
31
+ ".py",
32
+ ".go",
33
+ ".rs",
34
+ ".java",
35
+ ".c",
36
+ ".cc",
37
+ ".cpp",
38
+ ".h",
39
+ ".hpp",
40
+ ".rb",
41
+ ".sh",
42
+ ".swift",
43
+ ".kt",
44
+ ]);
45
+ /**
46
+ * Classifies a raw `--input`/`/rlm` argument into an {@link InputSource}:
47
+ * - `knowledge:<client-slug>` -> `{ type: "knowledge", clientSlug }`.
48
+ * - `http://...` / `https://...` -> `{ type: "url", url }`.
49
+ * - anything containing glob metacharacters (`* ? { } [ ] !`) -> `{ type: "glob", pattern, cwd }`.
50
+ * - everything else -> `{ type: "file", path }`, resolved against `cwd` if relative.
51
+ */
52
+ export function parseInputArg(input, cwd) {
53
+ const trimmed = input.trim();
54
+ if (trimmed.startsWith(KNOWLEDGE_PREFIX)) {
55
+ const clientSlug = trimmed.slice(KNOWLEDGE_PREFIX.length).trim();
56
+ return { type: "knowledge", clientSlug };
57
+ }
58
+ if (/^https?:\/\//i.test(trimmed)) {
59
+ return { type: "url", url: trimmed };
60
+ }
61
+ if (GLOB_CHARS.test(trimmed)) {
62
+ return { type: "glob", pattern: trimmed, cwd };
63
+ }
64
+ return { type: "file", path: path.isAbsolute(trimmed) ? trimmed : path.join(cwd, trimmed) };
65
+ }
66
+ /** Loads the given {@link InputSource}, returning its content and the context type to advertise to the RLM root loop. */
67
+ export async function loadInput(source) {
68
+ switch (source.type) {
69
+ case "file":
70
+ return loadFile(source.path);
71
+ case "glob":
72
+ return loadGlob(source.pattern, source.cwd);
73
+ case "url":
74
+ return loadUrl(source.url);
75
+ case "knowledge":
76
+ return loadKnowledge(source.clientSlug);
77
+ default: {
78
+ // Exhaustiveness check -- widen InputSource, forget a branch, get a compile error here.
79
+ const _never = source;
80
+ throw new Error(`loadInput: unknown source type "${_never.type}"`);
81
+ }
82
+ }
83
+ }
84
+ async function loadFile(filePath) {
85
+ const content = await readFile(filePath, "utf-8");
86
+ return { content, contextType: inferContextType(filePath) };
87
+ }
88
+ async function loadGlob(pattern, cwd) {
89
+ const matches = [];
90
+ for await (const match of glob(pattern, { cwd })) {
91
+ matches.push(match);
92
+ }
93
+ matches.sort();
94
+ if (matches.length === 0) {
95
+ throw new Error(`loadInput: glob pattern "${pattern}" matched no files (cwd: ${cwd})`);
96
+ }
97
+ const contextTypes = new Set();
98
+ const parts = await Promise.all(matches.map(async (relPath) => {
99
+ const absPath = path.isAbsolute(relPath) ? relPath : path.join(cwd, relPath);
100
+ const content = await readFile(absPath, "utf-8");
101
+ contextTypes.add(inferContextType(absPath));
102
+ return `--- source: ${absPath} ---\n${content}`;
103
+ }));
104
+ // A single matched file's context type carries through; a mixed match falls back to "text".
105
+ const contextType = contextTypes.size === 1 ? [...contextTypes][0] : "text";
106
+ return { content: parts.join("\n\n"), contextType };
107
+ }
108
+ async function loadUrl(url) {
109
+ const response = await fetch(url);
110
+ if (!response.ok) {
111
+ throw new Error(`loadInput: fetching "${url}" failed with status ${response.status}`);
112
+ }
113
+ const content = await response.text();
114
+ const contentType = response.headers.get("content-type") ?? "";
115
+ return { content, contextType: contentType.includes("json") ? "json" : "text" };
116
+ }
117
+ /** Chunks per section pulled from the client's knowledge store -- generous enough to cover most real stores. */
118
+ const KNOWLEDGE_SEARCH_LIMIT = 20;
119
+ async function loadKnowledge(clientSlug) {
120
+ const { KnowledgeManager } = await import("@draht/knowledge");
121
+ const manager = new KnowledgeManager(clientSlug);
122
+ try {
123
+ const [conventions, decisions] = await Promise.all([
124
+ manager.searchKnowledge("project conventions and AGENTS.md guidance", "general", KNOWLEDGE_SEARCH_LIMIT),
125
+ manager.searchKnowledge("past decisions and rationale", "decide", KNOWLEDGE_SEARCH_LIMIT),
126
+ ]);
127
+ const sections = [
128
+ formatKnowledgeSection("AGENTS.md / conventions", conventions),
129
+ formatKnowledgeSection("Past decisions", decisions),
130
+ ].filter((section) => section !== null);
131
+ if (sections.length === 0) {
132
+ throw new Error(`loadInput: no knowledge found for client "${clientSlug}"`);
133
+ }
134
+ return { content: sections.join("\n\n"), contextType: "text" };
135
+ }
136
+ finally {
137
+ manager.close();
138
+ }
139
+ }
140
+ function formatKnowledgeSection(title, results) {
141
+ if (results.length === 0)
142
+ return null;
143
+ const body = results.map((r) => r.content).join("\n\n");
144
+ return `## ${title}\n\n${body}`;
145
+ }
146
+ function inferContextType(filePath) {
147
+ const ext = path.extname(filePath).toLowerCase();
148
+ if (ext === ".json")
149
+ return "json";
150
+ if (ext === ".log")
151
+ return "logs";
152
+ if (CODE_EXTENSIONS.has(ext))
153
+ return "code";
154
+ return "text";
155
+ }
156
+ //# sourceMappingURL=loaders.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"loaders.js","sourceRoot":"","sources":["../src/loaders.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;GAYG;AAEH,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAClD,OAAO,KAAK,IAAI,MAAM,WAAW,CAAC;AAgBlC;;;;GAIG;AACH,MAAM,gBAAgB,GAAG,YAAY,CAAC;AAEtC,kGAAkG;AAClG,MAAM,UAAU,GAAG,YAAY,CAAC;AAEhC,MAAM,eAAe,GAAG,IAAI,GAAG,CAAC;IAC/B,KAAK;IACL,MAAM;IACN,KAAK;IACL,MAAM;IACN,MAAM;IACN,MAAM;IACN,KAAK;IACL,KAAK;IACL,KAAK;IACL,OAAO;IACP,IAAI;IACJ,KAAK;IACL,MAAM;IACN,IAAI;IACJ,MAAM;IACN,KAAK;IACL,KAAK;IACL,QAAQ;IACR,KAAK;CACL,CAAC,CAAC;AAEH;;;;;;GAMG;AACH,MAAM,UAAU,aAAa,CAAC,KAAa,EAAE,GAAW;IACvD,MAAM,OAAO,GAAG,KAAK,CAAC,IAAI,EAAE,CAAC;IAE7B,IAAI,OAAO,CAAC,UAAU,CAAC,gBAAgB,CAAC,EAAE,CAAC;QAC1C,MAAM,UAAU,GAAG,OAAO,CAAC,KAAK,CAAC,gBAAgB,CAAC,MAAM,CAAC,CAAC,IAAI,EAAE,CAAC;QACjE,OAAO,EAAE,IAAI,EAAE,WAAW,EAAE,UAAU,EAAE,CAAC;IAC1C,CAAC;IAED,IAAI,eAAe,CAAC,IAAI,CAAC,OAAO,CAAC,EAAE,CAAC;QACnC,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,GAAG,EAAE,OAAO,EAAE,CAAC;IACtC,CAAC;IAED,IAAI,UAAU,CAAC,IAAI,CAAC,OAAO,CAAC,EAAE,CAAC;QAC9B,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,OAAO,EAAE,GAAG,EAAE,CAAC;IAChD,CAAC;IAED,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,IAAI,CAAC,UAAU,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,EAAE,OAAO,CAAC,EAAE,CAAC;AAC7F,CAAC;AAED,yHAAyH;AACzH,MAAM,CAAC,KAAK,UAAU,SAAS,CAAC,MAAmB;IAClD,QAAQ,MAAM,CAAC,IAAI,EAAE,CAAC;QACrB,KAAK,MAAM;YACV,OAAO,QAAQ,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;QAC9B,KAAK,MAAM;YACV,OAAO,QAAQ,CAAC,MAAM,CAAC,OAAO,EAAE,MAAM,CAAC,GAAG,CAAC,CAAC;QAC7C,KAAK,KAAK;YACT,OAAO,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;QAC5B,KAAK,WAAW;YACf,OAAO,aAAa,CAAC,MAAM,CAAC,UAAU,CAAC,CAAC;QACzC,SAAS,CAAC;YACT,wFAAwF;YACxF,MAAM,MAAM,GAAU,MAAM,CAAC;YAC7B,MAAM,IAAI,KAAK,CAAC,mCAAoC,MAAsB,CAAC,IAAI,GAAG,CAAC,CAAC;QACrF,CAAC;IACF,CAAC;AACF,CAAC;AAED,KAAK,UAAU,QAAQ,CAAC,QAAgB;IACvC,MAAM,OAAO,GAAG,MAAM,QAAQ,CAAC,QAAQ,EAAE,OAAO,CAAC,CAAC;IAClD,OAAO,EAAE,OAAO,EAAE,WAAW,EAAE,gBAAgB,CAAC,QAAQ,CAAC,EAAE,CAAC;AAC7D,CAAC;AAED,KAAK,UAAU,QAAQ,CAAC,OAAe,EAAE,GAAW;IACnD,MAAM,OAAO,GAAa,EAAE,CAAC;IAC7B,IAAI,KAAK,EAAE,MAAM,KAAK,IAAI,IAAI,CAAC,OAAO,EAAE,EAAE,GAAG,EAAE,CAAC,EAAE,CAAC;QAClD,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;IACrB,CAAC;IACD,OAAO,CAAC,IAAI,EAAE,CAAC;IAEf,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC1B,MAAM,IAAI,KAAK,CAAC,4BAA4B,OAAO,4BAA4B,GAAG,GAAG,CAAC,CAAC;IACxF,CAAC;IAED,MAAM,YAAY,GAAG,IAAI,GAAG,EAAU,CAAC;IACvC,MAAM,KAAK,GAAG,MAAM,OAAO,CAAC,GAAG,CAC9B,OAAO,CAAC,GAAG,CAAC,KAAK,EAAE,OAAO,EAAE,EAAE;QAC7B,MAAM,OAAO,GAAG,IAAI,CAAC,UAAU,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,EAAE,OAAO,CAAC,CAAC;QAC7E,MAAM,OAAO,GAAG,MAAM,QAAQ,CAAC,OAAO,EAAE,OAAO,CAAC,CAAC;QACjD,YAAY,CAAC,GAAG,CAAC,gBAAgB,CAAC,OAAO,CAAC,CAAC,CAAC;QAC5C,OAAO,eAAe,OAAO,SAAS,OAAO,EAAE,CAAC;IACjD,CAAC,CAAC,CACF,CAAC;IAEF,4FAA4F;IAC5F,MAAM,WAAW,GAAG,YAAY,CAAC,IAAI,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,GAAG,YAAY,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC;IAC5E,OAAO,EAAE,OAAO,EAAE,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,WAAW,EAAE,CAAC;AACrD,CAAC;AAED,KAAK,UAAU,OAAO,CAAC,GAAW;IACjC,MAAM,QAAQ,GAAG,MAAM,KAAK,CAAC,GAAG,CAAC,CAAC;IAClC,IAAI,CAAC,QAAQ,CAAC,EAAE,EAAE,CAAC;QAClB,MAAM,IAAI,KAAK,CAAC,wBAAwB,GAAG,wBAAwB,QAAQ,CAAC,MAAM,EAAE,CAAC,CAAC;IACvF,CAAC;IACD,MAAM,OAAO,GAAG,MAAM,QAAQ,CAAC,IAAI,EAAE,CAAC;IACtC,MAAM,WAAW,GAAG,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC,cAAc,CAAC,IAAI,EAAE,CAAC;IAC/D,OAAO,EAAE,OAAO,EAAE,WAAW,EAAE,WAAW,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC;AACjF,CAAC;AAED,gHAAgH;AAChH,MAAM,sBAAsB,GAAG,EAAE,CAAC;AAElC,KAAK,UAAU,aAAa,CAAC,UAAkB;IAC9C,MAAM,EAAE,gBAAgB,EAAE,GAAG,MAAM,MAAM,CAAC,kBAAkB,CAAC,CAAC;IAC9D,MAAM,OAAO,GAAG,IAAI,gBAAgB,CAAC,UAAU,CAAC,CAAC;IACjD,IAAI,CAAC;QACJ,MAAM,CAAC,WAAW,EAAE,SAAS,CAAC,GAAG,MAAM,OAAO,CAAC,GAAG,CAAC;YAClD,OAAO,CAAC,eAAe,CAAC,4CAA4C,EAAE,SAAS,EAAE,sBAAsB,CAAC;YACxG,OAAO,CAAC,eAAe,CAAC,8BAA8B,EAAE,QAAQ,EAAE,sBAAsB,CAAC;SACzF,CAAC,CAAC;QAEH,MAAM,QAAQ,GAAG;YAChB,sBAAsB,CAAC,yBAAyB,EAAE,WAAW,CAAC;YAC9D,sBAAsB,CAAC,gBAAgB,EAAE,SAAS,CAAC;SACnD,CAAC,MAAM,CAAC,CAAC,OAAO,EAAqB,EAAE,CAAC,OAAO,KAAK,IAAI,CAAC,CAAC;QAE3D,IAAI,QAAQ,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YAC3B,MAAM,IAAI,KAAK,CAAC,6CAA6C,UAAU,GAAG,CAAC,CAAC;QAC7E,CAAC;QAED,OAAO,EAAE,OAAO,EAAE,QAAQ,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,WAAW,EAAE,MAAM,EAAE,CAAC;IAChE,CAAC;YAAS,CAAC;QACV,OAAO,CAAC,KAAK,EAAE,CAAC;IACjB,CAAC;AACF,CAAC;AAED,SAAS,sBAAsB,CAAC,KAAa,EAAE,OAAuB;IACrE,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,IAAI,CAAC;IACtC,MAAM,IAAI,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;IACxD,OAAO,MAAM,KAAK,OAAO,IAAI,EAAE,CAAC;AACjC,CAAC;AAED,SAAS,gBAAgB,CAAC,QAAgB;IACzC,MAAM,GAAG,GAAG,IAAI,CAAC,OAAO,CAAC,QAAQ,CAAC,CAAC,WAAW,EAAE,CAAC;IACjD,IAAI,GAAG,KAAK,OAAO;QAAE,OAAO,MAAM,CAAC;IACnC,IAAI,GAAG,KAAK,MAAM;QAAE,OAAO,MAAM,CAAC;IAClC,IAAI,eAAe,CAAC,GAAG,CAAC,GAAG,CAAC;QAAE,OAAO,MAAM,CAAC;IAC5C,OAAO,MAAM,CAAC;AACf,CAAC","sourcesContent":["/**\n * Input loaders -- turn a raw `--input`/`/rlm <input>` argument into loaded\n * text content ready to become an `RlmSession`'s `prompt`. Covers four kinds\n * of source: a single file, a glob pattern, an `http(s)://` URL, and a named\n * client knowledge base (via `@draht/knowledge`). See\n * .planning/phases/29-agent-cli-integration/29-01-PLAN.md, Architecture\n * section 1.\n *\n * `parseInputArg` classifies the raw string; `loadInput` does the actual\n * I/O. Kept as two functions so callers (CLI, `/rlm` command, `rlm_query`\n * tool) can validate/echo back what kind of source they resolved before\n * paying the cost of loading it.\n */\n\nimport { glob, readFile } from \"node:fs/promises\";\nimport * as path from \"node:path\";\nimport type { SearchResult } from \"@draht/knowledge\";\n\nexport type InputSource =\n\t| { type: \"file\"; path: string }\n\t| { type: \"glob\"; pattern: string; cwd: string }\n\t| { type: \"url\"; url: string }\n\t| { type: \"knowledge\"; clientSlug: string };\n\nexport interface LoadedInput {\n\t/** The loaded/concatenated text content. */\n\tcontent: string;\n\t/** What kind of content `content` holds (e.g. \"text\", \"code\", \"logs\", \"json\") -- see `PromptVars.contextType`. */\n\tcontextType: string;\n}\n\n/**\n * Recognized prefix for the knowledge-base input syntax: `knowledge:<client-slug>`\n * (e.g. `knowledge:acme-corp`), where `<client-slug>` matches `ClientConfig.slug`\n * from `@draht/knowledge`'s `client-config.ts`.\n */\nconst KNOWLEDGE_PREFIX = \"knowledge:\";\n\n/** Characters that mark an `--input` string as a glob pattern rather than a literal file path. */\nconst GLOB_CHARS = /[*?{}[\\]!]/;\n\nconst CODE_EXTENSIONS = new Set([\n\t\".ts\",\n\t\".tsx\",\n\t\".js\",\n\t\".jsx\",\n\t\".mjs\",\n\t\".cjs\",\n\t\".py\",\n\t\".go\",\n\t\".rs\",\n\t\".java\",\n\t\".c\",\n\t\".cc\",\n\t\".cpp\",\n\t\".h\",\n\t\".hpp\",\n\t\".rb\",\n\t\".sh\",\n\t\".swift\",\n\t\".kt\",\n]);\n\n/**\n * Classifies a raw `--input`/`/rlm` argument into an {@link InputSource}:\n * - `knowledge:<client-slug>` -> `{ type: \"knowledge\", clientSlug }`.\n * - `http://...` / `https://...` -> `{ type: \"url\", url }`.\n * - anything containing glob metacharacters (`* ? { } [ ] !`) -> `{ type: \"glob\", pattern, cwd }`.\n * - everything else -> `{ type: \"file\", path }`, resolved against `cwd` if relative.\n */\nexport function parseInputArg(input: string, cwd: string): InputSource {\n\tconst trimmed = input.trim();\n\n\tif (trimmed.startsWith(KNOWLEDGE_PREFIX)) {\n\t\tconst clientSlug = trimmed.slice(KNOWLEDGE_PREFIX.length).trim();\n\t\treturn { type: \"knowledge\", clientSlug };\n\t}\n\n\tif (/^https?:\\/\\//i.test(trimmed)) {\n\t\treturn { type: \"url\", url: trimmed };\n\t}\n\n\tif (GLOB_CHARS.test(trimmed)) {\n\t\treturn { type: \"glob\", pattern: trimmed, cwd };\n\t}\n\n\treturn { type: \"file\", path: path.isAbsolute(trimmed) ? trimmed : path.join(cwd, trimmed) };\n}\n\n/** Loads the given {@link InputSource}, returning its content and the context type to advertise to the RLM root loop. */\nexport async function loadInput(source: InputSource): Promise<LoadedInput> {\n\tswitch (source.type) {\n\t\tcase \"file\":\n\t\t\treturn loadFile(source.path);\n\t\tcase \"glob\":\n\t\t\treturn loadGlob(source.pattern, source.cwd);\n\t\tcase \"url\":\n\t\t\treturn loadUrl(source.url);\n\t\tcase \"knowledge\":\n\t\t\treturn loadKnowledge(source.clientSlug);\n\t\tdefault: {\n\t\t\t// Exhaustiveness check -- widen InputSource, forget a branch, get a compile error here.\n\t\t\tconst _never: never = source;\n\t\t\tthrow new Error(`loadInput: unknown source type \"${(_never as InputSource).type}\"`);\n\t\t}\n\t}\n}\n\nasync function loadFile(filePath: string): Promise<LoadedInput> {\n\tconst content = await readFile(filePath, \"utf-8\");\n\treturn { content, contextType: inferContextType(filePath) };\n}\n\nasync function loadGlob(pattern: string, cwd: string): Promise<LoadedInput> {\n\tconst matches: string[] = [];\n\tfor await (const match of glob(pattern, { cwd })) {\n\t\tmatches.push(match);\n\t}\n\tmatches.sort();\n\n\tif (matches.length === 0) {\n\t\tthrow new Error(`loadInput: glob pattern \"${pattern}\" matched no files (cwd: ${cwd})`);\n\t}\n\n\tconst contextTypes = new Set<string>();\n\tconst parts = await Promise.all(\n\t\tmatches.map(async (relPath) => {\n\t\t\tconst absPath = path.isAbsolute(relPath) ? relPath : path.join(cwd, relPath);\n\t\t\tconst content = await readFile(absPath, \"utf-8\");\n\t\t\tcontextTypes.add(inferContextType(absPath));\n\t\t\treturn `--- source: ${absPath} ---\\n${content}`;\n\t\t}),\n\t);\n\n\t// A single matched file's context type carries through; a mixed match falls back to \"text\".\n\tconst contextType = contextTypes.size === 1 ? [...contextTypes][0] : \"text\";\n\treturn { content: parts.join(\"\\n\\n\"), contextType };\n}\n\nasync function loadUrl(url: string): Promise<LoadedInput> {\n\tconst response = await fetch(url);\n\tif (!response.ok) {\n\t\tthrow new Error(`loadInput: fetching \"${url}\" failed with status ${response.status}`);\n\t}\n\tconst content = await response.text();\n\tconst contentType = response.headers.get(\"content-type\") ?? \"\";\n\treturn { content, contextType: contentType.includes(\"json\") ? \"json\" : \"text\" };\n}\n\n/** Chunks per section pulled from the client's knowledge store -- generous enough to cover most real stores. */\nconst KNOWLEDGE_SEARCH_LIMIT = 20;\n\nasync function loadKnowledge(clientSlug: string): Promise<LoadedInput> {\n\tconst { KnowledgeManager } = await import(\"@draht/knowledge\");\n\tconst manager = new KnowledgeManager(clientSlug);\n\ttry {\n\t\tconst [conventions, decisions] = await Promise.all([\n\t\t\tmanager.searchKnowledge(\"project conventions and AGENTS.md guidance\", \"general\", KNOWLEDGE_SEARCH_LIMIT),\n\t\t\tmanager.searchKnowledge(\"past decisions and rationale\", \"decide\", KNOWLEDGE_SEARCH_LIMIT),\n\t\t]);\n\n\t\tconst sections = [\n\t\t\tformatKnowledgeSection(\"AGENTS.md / conventions\", conventions),\n\t\t\tformatKnowledgeSection(\"Past decisions\", decisions),\n\t\t].filter((section): section is string => section !== null);\n\n\t\tif (sections.length === 0) {\n\t\t\tthrow new Error(`loadInput: no knowledge found for client \"${clientSlug}\"`);\n\t\t}\n\n\t\treturn { content: sections.join(\"\\n\\n\"), contextType: \"text\" };\n\t} finally {\n\t\tmanager.close();\n\t}\n}\n\nfunction formatKnowledgeSection(title: string, results: SearchResult[]): string | null {\n\tif (results.length === 0) return null;\n\tconst body = results.map((r) => r.content).join(\"\\n\\n\");\n\treturn `## ${title}\\n\\n${body}`;\n}\n\nfunction inferContextType(filePath: string): string {\n\tconst ext = path.extname(filePath).toLowerCase();\n\tif (ext === \".json\") return \"json\";\n\tif (ext === \".log\") return \"logs\";\n\tif (CODE_EXTENSIONS.has(ext)) return \"code\";\n\treturn \"text\";\n}\n"]}