@draht/rlm 2026.7.12-1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +167 -0
- package/dist/cost-comparison.d.ts +81 -0
- package/dist/cost-comparison.d.ts.map +1 -0
- package/dist/cost-comparison.js +93 -0
- package/dist/cost-comparison.js.map +1 -0
- package/dist/index.d.ts +11 -0
- package/dist/index.d.ts.map +1 -0
- package/dist/index.js +14 -0
- package/dist/index.js.map +1 -0
- package/dist/loaders.d.ts +44 -0
- package/dist/loaders.d.ts.map +1 -0
- package/dist/loaders.js +156 -0
- package/dist/loaders.js.map +1 -0
- package/dist/prompts.d.ts +49 -0
- package/dist/prompts.d.ts.map +1 -0
- package/dist/prompts.js +71 -0
- package/dist/prompts.js.map +1 -0
- package/dist/router-session.d.ts +50 -0
- package/dist/router-session.d.ts.map +1 -0
- package/dist/router-session.js +251 -0
- package/dist/router-session.js.map +1 -0
- package/dist/sandbox.d.ts +95 -0
- package/dist/sandbox.d.ts.map +1 -0
- package/dist/sandbox.js +273 -0
- package/dist/sandbox.js.map +1 -0
- package/dist/session.d.ts +162 -0
- package/dist/session.d.ts.map +1 -0
- package/dist/session.js +803 -0
- package/dist/session.js.map +1 -0
- package/dist/trajectory.d.ts +71 -0
- package/dist/trajectory.d.ts.map +1 -0
- package/dist/trajectory.js +65 -0
- package/dist/trajectory.js.map +1 -0
- package/dist/types.d.ts +140 -0
- package/dist/types.d.ts.map +1 -0
- package/dist/types.js +8 -0
- package/dist/types.js.map +1 -0
- package/package.json +35 -0
- package/python/repl_driver.py +744 -0
- package/sandbox/macos.sb +89 -0
package/README.md
ADDED
|
@@ -0,0 +1,167 @@
|
|
|
1
|
+
# @draht/rlm
|
|
2
|
+
|
|
3
|
+
Recursive Language Models (RLM) — the inference-time scaffold from Zhang,
|
|
4
|
+
Kraska, Khattab (2026), *Recursive Language Models* (arXiv:2512.24601):
|
|
5
|
+
root-LLM-produces-code → REPL-executes → truncated-stdout → history-append →
|
|
6
|
+
FINAL-check. Lets a root model answer questions about inputs far larger than
|
|
7
|
+
its own context window by writing Python that inspects, chunks, and searches
|
|
8
|
+
the content from inside a sandboxed REPL, optionally recursing into
|
|
9
|
+
sub-`llm_query` calls, instead of the caller ever loading the raw content
|
|
10
|
+
into a single prompt.
|
|
11
|
+
|
|
12
|
+
## When to prefer RLM over a normal agent read
|
|
13
|
+
|
|
14
|
+
Reach for RLM when the input genuinely doesn't fit the model's context
|
|
15
|
+
window, or fits only by truncating away content that might matter:
|
|
16
|
+
|
|
17
|
+
- A log file, dataset dump, or exported knowledge base that's tens or
|
|
18
|
+
hundreds of megabytes — orders of magnitude past any model's window.
|
|
19
|
+
- A directory/glob of source files too large to read whole without evicting
|
|
20
|
+
earlier conversation context.
|
|
21
|
+
- Any case where you'd otherwise have to guess which prefix/suffix/sample of
|
|
22
|
+
the input to truncate to, and a wrong guess means missing the answer.
|
|
23
|
+
|
|
24
|
+
Don't reach for RLM when the input already fits comfortably in the model's
|
|
25
|
+
context window — a normal read is faster, cheaper, and easier to debug. RLM
|
|
26
|
+
trades one direct read for several extra model round trips (see below); that
|
|
27
|
+
trade only pays off when the alternative is truncation-induced information
|
|
28
|
+
loss, not just "this file is somewhat long."
|
|
29
|
+
|
|
30
|
+
## Cost envelope
|
|
31
|
+
|
|
32
|
+
A normal read is one call: the whole input goes into the prompt, the model
|
|
33
|
+
answers once. RLM is **not** universally cheaper than that — it's a
|
|
34
|
+
different trade:
|
|
35
|
+
|
|
36
|
+
- The root model makes one call per REPL step (a handful to a few dozen
|
|
37
|
+
steps for a typical run), and each step's Python may itself call
|
|
38
|
+
`llm_query(...)`, which is a *second* model call (typically to a smaller
|
|
39
|
+
"rlm-sub" role — see `router-session.ts`'s `RlmRole`) that also costs
|
|
40
|
+
money.
|
|
41
|
+
- So RLM's total cost is `sum(root-step costs) + sum(sub-call costs)` versus
|
|
42
|
+
a baseline's single call. RLM wins when the baseline's single call would
|
|
43
|
+
otherwise require either failing outright (input doesn't fit at all) or
|
|
44
|
+
paying for a much larger context window / re-reading the input multiple
|
|
45
|
+
times across a conversation to compensate for truncation — not in the
|
|
46
|
+
general case.
|
|
47
|
+
- **Don't guess — measure.** `src/cost-comparison.ts`'s `compareCost()`
|
|
48
|
+
computes both numbers for one trajectory: `rlmCostUsd` from the trajectory's
|
|
49
|
+
own logged cost entries (`readCostLog`/`getSessionCosts` from
|
|
50
|
+
`@draht/router`, filtered by `trajectoryId`), and `baselineCostUsd` as the
|
|
51
|
+
estimated cost (via `@draht/router`'s `estimateCost`) of a single call that
|
|
52
|
+
truncates the same content down to the target model's context window
|
|
53
|
+
(first-N-characters-that-fit — a simple, named strategy, not a claim that
|
|
54
|
+
it's the smartest possible truncation). Run it against a real trajectory ID
|
|
55
|
+
before assuming RLM is the cheaper (or more expensive) option for your
|
|
56
|
+
specific input/model pair; `.draht/rlm/cost-comparison-report.json` is
|
|
57
|
+
where a comparison run writes its report.
|
|
58
|
+
|
|
59
|
+
## Observability: trajectory logs & replay
|
|
60
|
+
|
|
61
|
+
Every `createRouterBackedSession` run writes a newline-delimited JSON
|
|
62
|
+
trajectory log to `.draht/rlm/<trajectoryId>.jsonl` (`src/trajectory.ts`) —
|
|
63
|
+
one `TrajectoryStepEntry` per REPL step (its code, truncated stdout, error if
|
|
64
|
+
any, the sub-calls it triggered, and that step's own cost) plus one terminal
|
|
65
|
+
`TrajectoryFinalEntry` once the session resolves. `trajectoryId` is the same
|
|
66
|
+
ID already used to tag the session's `@draht/router` cost-log entries, so a
|
|
67
|
+
trajectory's logs and its cost accounting always correlate by the same key.
|
|
68
|
+
|
|
69
|
+
Because the whole run is on disk, it can be reconstructed with **zero LLM
|
|
70
|
+
calls**:
|
|
71
|
+
|
|
72
|
+
```bash
|
|
73
|
+
draht rlm replay <trajectory-id>
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
reads the JSONL log via `readTrajectory()` and prints the final answer (add a
|
|
77
|
+
verbose flag for the full step-by-step trace) — no router, no model, no
|
|
78
|
+
network access on that code path at all. Useful for auditing what a session
|
|
79
|
+
actually did, debugging a run that didn't reach `FINAL`, or reviewing cost
|
|
80
|
+
without re-running anything.
|
|
81
|
+
|
|
82
|
+
The `test/s-niah.test.ts` regression suite exercises recall at 10x and 100x a
|
|
83
|
+
mocked root model's context window (synthetic needle-in-haystack at ~4M and
|
|
84
|
+
~40M characters, against a scripted mock `rootLlm` — no real model calls) as
|
|
85
|
+
a standing check that the scaffold still finds a planted needle at scale.
|
|
86
|
+
|
|
87
|
+
## Worked example
|
|
88
|
+
|
|
89
|
+
Say `deploy.log` is a 40 MB deployment log and you need to know what caused
|
|
90
|
+
an outage buried somewhere in it — too large to paste into a normal prompt.
|
|
91
|
+
The same question, three ways:
|
|
92
|
+
|
|
93
|
+
**Interactive `/rlm` command** (inside a `draht` session):
|
|
94
|
+
|
|
95
|
+
```
|
|
96
|
+
/rlm deploy.log What caused the outage around the 03:00 UTC deploy?
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
**`draht rlm` CLI** (scriptable, one-shot):
|
|
100
|
+
|
|
101
|
+
```bash
|
|
102
|
+
draht rlm --input ./deploy.log --query "What caused the outage around the 03:00 UTC deploy?"
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
**`rlm_query` tool** (an agent defers the read itself, mid-session, instead
|
|
106
|
+
of loading the log into its own context):
|
|
107
|
+
|
|
108
|
+
```json
|
|
109
|
+
{
|
|
110
|
+
"tool": "rlm_query",
|
|
111
|
+
"input": {
|
|
112
|
+
"input": "deploy.log",
|
|
113
|
+
"query": "What caused the outage around the 03:00 UTC deploy?"
|
|
114
|
+
}
|
|
115
|
+
}
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
All three share the same path under the hood: `parseInputArg`/`loadInput`
|
|
119
|
+
resolve `deploy.log` into content, `createRouterBackedSession` wraps it as
|
|
120
|
+
the REPL's `context` with the question prepended as a header, and the root
|
|
121
|
+
model writes Python to grep/chunk/search `context` — recursing into
|
|
122
|
+
`llm_query` sub-calls for individual chunks as needed — until it calls
|
|
123
|
+
`FINAL(...)`/`FINAL_VAR(...)` with the answer. `--input`/`input` also accepts
|
|
124
|
+
a glob (`"src/**/*.ts"`), an `http(s)://` URL, or `knowledge:<client-slug>`
|
|
125
|
+
for a named client knowledge base.
|
|
126
|
+
|
|
127
|
+
To check whether that particular run was worth it cost-wise, take the
|
|
128
|
+
`trajectoryId` of its `.draht/rlm/<trajectoryId>.jsonl` log (or the matching
|
|
129
|
+
`trajectoryId` field in `.draht/cost-log.jsonl`) and feed it into
|
|
130
|
+
`compareCost()` (see Cost envelope, above) rather than assuming either
|
|
131
|
+
direction.
|
|
132
|
+
|
|
133
|
+
## Development
|
|
134
|
+
|
|
135
|
+
```bash
|
|
136
|
+
npm install # from repo root, resolves this workspace package
|
|
137
|
+
cd packages/rlm
|
|
138
|
+
npx tsc --noEmit # typecheck
|
|
139
|
+
npm test # vitest
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
## Package layout
|
|
143
|
+
|
|
144
|
+
- `src/session.ts` — `RlmSession` root loop (step/run, `FINAL`/`FINAL_VAR`
|
|
145
|
+
sentinels, constant-size history).
|
|
146
|
+
- `src/router-session.ts` — `createRouterBackedSession`, the production
|
|
147
|
+
`@draht/router`-backed factory (prompt-tier selection, cost logging,
|
|
148
|
+
trajectory logging).
|
|
149
|
+
- `src/trajectory.ts` — trajectory JSONL append/read (this phase).
|
|
150
|
+
- `src/cost-comparison.ts` — RLM-vs-baseline cost harness (this phase).
|
|
151
|
+
- `src/loaders.ts` — `--input`/`/rlm <input>` source loaders (file, glob,
|
|
152
|
+
URL, `knowledge:<slug>`).
|
|
153
|
+
- `src/prompts.ts` — tiered system-prompt templates, selected by the root
|
|
154
|
+
model's context window.
|
|
155
|
+
- `python/repl_driver.py` — the sandboxed Python REPL driver.
|
|
156
|
+
- `packages/coding-agent/src/rlm-cli.ts` — `draht rlm` CLI subcommand.
|
|
157
|
+
- `packages/rlm-agent/` — `/rlm` command + `rlm_query` tool extension for
|
|
158
|
+
normal agent sessions.
|
|
159
|
+
|
|
160
|
+
## AGENTS.md
|
|
161
|
+
|
|
162
|
+
RLM is a cross-cutting coding-agent capability, not a property of any single
|
|
163
|
+
generated project's stack — none of `packages/templates/src/*.md`'s
|
|
164
|
+
per-stack templates (Astro, Go/gRPC, SST TypeScript) is the right home for a
|
|
165
|
+
"when to use RLM" note, and this repo's own root `AGENTS.md` is a
|
|
166
|
+
contributor-workflow doc, not a feature reference. This README is the
|
|
167
|
+
primary documentation surface for RLM; no template changes were made.
|
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Cost-comparison harness: RLM trajectory cost vs. a truncate-and-single-call
|
|
3
|
+
* baseline for the same task.
|
|
4
|
+
*
|
|
5
|
+
* `rlmCostUsd` is the real, already-incurred cost of a trajectory recovered
|
|
6
|
+
* from `@draht/router`'s cost log (every root + sub call `createRouterBackedSession`
|
|
7
|
+
* logs is tagged with one `trajectoryId` -- see router-session.ts). There is
|
|
8
|
+
* nothing to "compare" about that half; it's just a sum-and-filter over
|
|
9
|
+
* already-logged `CostEntry` rows.
|
|
10
|
+
*
|
|
11
|
+
* `baselineCostUsd` is necessarily an *estimate*, not a measurement: it models
|
|
12
|
+
* the cost of the simplest plausible non-RLM alternative for an oversize
|
|
13
|
+
* task -- truncate the context down to whatever fits the target model's
|
|
14
|
+
* context window, then make one single call -- and prices that hypothetical
|
|
15
|
+
* call via `@draht/router`'s `estimateCost`. The specific truncation strategy
|
|
16
|
+
* modeled here is deliberately the crudest one: keep the first N characters
|
|
17
|
+
* of the context that fit the window (a `CHARS_PER_TOKEN` heuristic converts
|
|
18
|
+
* the token-denominated `contextWindow` into a character budget). A smarter
|
|
19
|
+
* baseline (e.g. picking the *most relevant* N characters via embedding
|
|
20
|
+
* search, or summarizing before truncating) would likely produce different
|
|
21
|
+
* numbers -- that's out of scope here. This harness answers "what would the
|
|
22
|
+
* cheapest naive single-call alternative have cost", not "what is the best
|
|
23
|
+
* possible non-RLM approach". See
|
|
24
|
+
* .planning/phases/30-eval-observability-docs/30-01-PLAN.md, Architecture
|
|
25
|
+
* section 3, and its Risks section.
|
|
26
|
+
*/
|
|
27
|
+
/** Which target model the truncate-and-single-call baseline is priced against. */
|
|
28
|
+
export interface BaselineModel {
|
|
29
|
+
provider: string;
|
|
30
|
+
modelName: string;
|
|
31
|
+
/** Context window, in tokens -- used to compute how much of the context the baseline call could fit after truncation. */
|
|
32
|
+
contextWindow: number;
|
|
33
|
+
}
|
|
34
|
+
/** Result of comparing one RLM trajectory's real cost against a truncate-and-single-call baseline for the same task. */
|
|
35
|
+
export interface CostComparisonResult {
|
|
36
|
+
/**
|
|
37
|
+
* Identifies which task/trajectory this comparison covers. There is no
|
|
38
|
+
* separate task-name concept tracked elsewhere in this codebase, so this
|
|
39
|
+
* is the `trajectoryId` itself -- callers that want a human-readable label
|
|
40
|
+
* should track that mapping themselves.
|
|
41
|
+
*/
|
|
42
|
+
task: string;
|
|
43
|
+
/** Sum of every cost-log entry (root call + every `llm_query` sub-call) tagged with this trajectoryId. */
|
|
44
|
+
rlmCostUsd: number;
|
|
45
|
+
/**
|
|
46
|
+
* Estimated cost of truncating the context to fit `model`'s window and
|
|
47
|
+
* making one single call instead (see the module doc for the exact
|
|
48
|
+
* truncation approach modeled).
|
|
49
|
+
*/
|
|
50
|
+
baselineCostUsd: number;
|
|
51
|
+
/** Number of root-LLM steps the RLM trajectory took (one "rlm-root"-role cost entry per step -- see router-session.ts's `rootLlm`). */
|
|
52
|
+
rlmSteps: number;
|
|
53
|
+
/** The full (untruncated) context length, in characters, the RLM trajectory operated over. */
|
|
54
|
+
contextLengthChars: number;
|
|
55
|
+
}
|
|
56
|
+
/**
|
|
57
|
+
* Compares an already-run RLM trajectory's real cost (read back from the
|
|
58
|
+
* cost log at `costLogPath`, filtered by `trajectoryId`) against the
|
|
59
|
+
* estimated cost of a truncate-and-single-call baseline for the same task:
|
|
60
|
+
* truncate `contextLengthChars` down to whatever fits `model.contextWindow`
|
|
61
|
+
* (first-N-characters, converted via `CHARS_PER_TOKEN`), then price one call
|
|
62
|
+
* to `model` via `@draht/router`'s `estimateCost`.
|
|
63
|
+
*
|
|
64
|
+
* Doesn't run anything itself -- the RLM trajectory must already have run
|
|
65
|
+
* (via `createRouterBackedSession`, logging its cost entries under
|
|
66
|
+
* `costLogPath`) before calling this; the baseline call is hypothetical and
|
|
67
|
+
* never actually dispatched. This function is pure arithmetic over its
|
|
68
|
+
* inputs plus the cost log, and makes no claim about which side is cheaper --
|
|
69
|
+
* that depends entirely on the task.
|
|
70
|
+
*/
|
|
71
|
+
export declare function compareCost(trajectoryId: string, contextLengthChars: number, model: BaselineModel, costLogPath?: string): CostComparisonResult;
|
|
72
|
+
/**
|
|
73
|
+
* Writes a `CostComparisonResult` to disk as the "eval report" R30-EVAL.3
|
|
74
|
+
* asks for, creating parent directories as needed. Defaults to
|
|
75
|
+
* `.draht/rlm/cost-comparison-report.json` (relative to `process.cwd()`),
|
|
76
|
+
* mirroring `trajectory.ts`'s `DEFAULT_LOG_DIR` convention -- callers that
|
|
77
|
+
* don't want to touch that real location (e.g. tests) should pass an
|
|
78
|
+
* explicit `reportPath` instead.
|
|
79
|
+
*/
|
|
80
|
+
export declare function writeCostComparisonReport(result: CostComparisonResult, reportPath?: string): void;
|
|
81
|
+
//# sourceMappingURL=cost-comparison.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"cost-comparison.d.ts","sourceRoot":"","sources":["../src/cost-comparison.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;GAyBG;AAMH,kFAAkF;AAClF,MAAM,WAAW,aAAa;IAC7B,QAAQ,EAAE,MAAM,CAAC;IACjB,SAAS,EAAE,MAAM,CAAC;IAClB,yHAAyH;IACzH,aAAa,EAAE,MAAM,CAAC;CACtB;AAED,wHAAwH;AACxH,MAAM,WAAW,oBAAoB;IACpC;;;;;OAKG;IACH,IAAI,EAAE,MAAM,CAAC;IACb,0GAA0G;IAC1G,UAAU,EAAE,MAAM,CAAC;IACnB;;;;OAIG;IACH,eAAe,EAAE,MAAM,CAAC;IACxB,uIAAuI;IACvI,QAAQ,EAAE,MAAM,CAAC;IACjB,8FAA8F;IAC9F,kBAAkB,EAAE,MAAM,CAAC;CAC3B;AAuBD;;;;;;;;;;;;;;GAcG;AACH,wBAAgB,WAAW,CAC1B,YAAY,EAAE,MAAM,EACpB,kBAAkB,EAAE,MAAM,EAC1B,KAAK,EAAE,aAAa,EACpB,WAAW,CAAC,EAAE,MAAM,GAClB,oBAAoB,CAwBtB;AAED;;;;;;;GAOG;AACH,wBAAgB,yBAAyB,CAAC,MAAM,EAAE,oBAAoB,EAAE,UAAU,CAAC,EAAE,MAAM,GAAG,IAAI,CAIjG"}
|
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Cost-comparison harness: RLM trajectory cost vs. a truncate-and-single-call
|
|
3
|
+
* baseline for the same task.
|
|
4
|
+
*
|
|
5
|
+
* `rlmCostUsd` is the real, already-incurred cost of a trajectory recovered
|
|
6
|
+
* from `@draht/router`'s cost log (every root + sub call `createRouterBackedSession`
|
|
7
|
+
* logs is tagged with one `trajectoryId` -- see router-session.ts). There is
|
|
8
|
+
* nothing to "compare" about that half; it's just a sum-and-filter over
|
|
9
|
+
* already-logged `CostEntry` rows.
|
|
10
|
+
*
|
|
11
|
+
* `baselineCostUsd` is necessarily an *estimate*, not a measurement: it models
|
|
12
|
+
* the cost of the simplest plausible non-RLM alternative for an oversize
|
|
13
|
+
* task -- truncate the context down to whatever fits the target model's
|
|
14
|
+
* context window, then make one single call -- and prices that hypothetical
|
|
15
|
+
* call via `@draht/router`'s `estimateCost`. The specific truncation strategy
|
|
16
|
+
* modeled here is deliberately the crudest one: keep the first N characters
|
|
17
|
+
* of the context that fit the window (a `CHARS_PER_TOKEN` heuristic converts
|
|
18
|
+
* the token-denominated `contextWindow` into a character budget). A smarter
|
|
19
|
+
* baseline (e.g. picking the *most relevant* N characters via embedding
|
|
20
|
+
* search, or summarizing before truncating) would likely produce different
|
|
21
|
+
* numbers -- that's out of scope here. This harness answers "what would the
|
|
22
|
+
* cheapest naive single-call alternative have cost", not "what is the best
|
|
23
|
+
* possible non-RLM approach". See
|
|
24
|
+
* .planning/phases/30-eval-observability-docs/30-01-PLAN.md, Architecture
|
|
25
|
+
* section 3, and its Risks section.
|
|
26
|
+
*/
|
|
27
|
+
import { mkdirSync, writeFileSync } from "node:fs";
|
|
28
|
+
import { dirname, resolve } from "node:path";
|
|
29
|
+
import { estimateCost, readCostLog } from "@draht/router";
|
|
30
|
+
/**
|
|
31
|
+
* Rough heuristic: ~4 characters per token for English-ish text. This is the
|
|
32
|
+
* same order-of-magnitude approximation used broadly across LLM tooling when
|
|
33
|
+
* an exact tokenizer isn't wired in for a given model -- it is NOT
|
|
34
|
+
* model-exact, and the `baselineCostUsd` this produces should be read as
|
|
35
|
+
* "same ballpark", not "precise". See the module doc above and the Risks
|
|
36
|
+
* section of the phase 30 plan.
|
|
37
|
+
*/
|
|
38
|
+
const CHARS_PER_TOKEN = 4;
|
|
39
|
+
/**
|
|
40
|
+
* Fixed assumed output length (tokens) for the baseline's single call -- a
|
|
41
|
+
* plausible "here's the answer" response size. Deliberately not modeled
|
|
42
|
+
* per-task: this harness needs a reasonable, explicitly-documented constant,
|
|
43
|
+
* not a task-specific prediction of how long an answer would be.
|
|
44
|
+
*/
|
|
45
|
+
const BASELINE_OUTPUT_TOKENS = 500;
|
|
46
|
+
/** Default path the "eval report" JSON is written to/read from, relative to `process.cwd()`. */
|
|
47
|
+
const DEFAULT_REPORT_PATH = ".draht/rlm/cost-comparison-report.json";
|
|
48
|
+
/**
|
|
49
|
+
* Compares an already-run RLM trajectory's real cost (read back from the
|
|
50
|
+
* cost log at `costLogPath`, filtered by `trajectoryId`) against the
|
|
51
|
+
* estimated cost of a truncate-and-single-call baseline for the same task:
|
|
52
|
+
* truncate `contextLengthChars` down to whatever fits `model.contextWindow`
|
|
53
|
+
* (first-N-characters, converted via `CHARS_PER_TOKEN`), then price one call
|
|
54
|
+
* to `model` via `@draht/router`'s `estimateCost`.
|
|
55
|
+
*
|
|
56
|
+
* Doesn't run anything itself -- the RLM trajectory must already have run
|
|
57
|
+
* (via `createRouterBackedSession`, logging its cost entries under
|
|
58
|
+
* `costLogPath`) before calling this; the baseline call is hypothetical and
|
|
59
|
+
* never actually dispatched. This function is pure arithmetic over its
|
|
60
|
+
* inputs plus the cost log, and makes no claim about which side is cheaper --
|
|
61
|
+
* that depends entirely on the task.
|
|
62
|
+
*/
|
|
63
|
+
export function compareCost(trajectoryId, contextLengthChars, model, costLogPath) {
|
|
64
|
+
const entries = readCostLog(costLogPath).filter((entry) => entry.trajectoryId === trajectoryId);
|
|
65
|
+
const rlmCostUsd = entries.reduce((sum, entry) => sum + entry.estimatedCostUsd, 0);
|
|
66
|
+
// Exactly one "rlm-root"-role cost entry is logged per root-LLM step (see
|
|
67
|
+
// router-session.ts's `logRouterCall("rlm-root", ...)` inside `rootLlm`) --
|
|
68
|
+
// sub-calls (role "rlm-sub") don't count as separate steps.
|
|
69
|
+
const rlmSteps = entries.filter((entry) => entry.role === "rlm-root").length;
|
|
70
|
+
const truncatedInputTokens = Math.min(model.contextWindow, Math.ceil(contextLengthChars / CHARS_PER_TOKEN));
|
|
71
|
+
const baselineCostUsd = estimateCost(model.provider, model.modelName, truncatedInputTokens, BASELINE_OUTPUT_TOKENS, 0);
|
|
72
|
+
return {
|
|
73
|
+
task: trajectoryId,
|
|
74
|
+
rlmCostUsd,
|
|
75
|
+
baselineCostUsd,
|
|
76
|
+
rlmSteps,
|
|
77
|
+
contextLengthChars,
|
|
78
|
+
};
|
|
79
|
+
}
|
|
80
|
+
/**
|
|
81
|
+
* Writes a `CostComparisonResult` to disk as the "eval report" R30-EVAL.3
|
|
82
|
+
* asks for, creating parent directories as needed. Defaults to
|
|
83
|
+
* `.draht/rlm/cost-comparison-report.json` (relative to `process.cwd()`),
|
|
84
|
+
* mirroring `trajectory.ts`'s `DEFAULT_LOG_DIR` convention -- callers that
|
|
85
|
+
* don't want to touch that real location (e.g. tests) should pass an
|
|
86
|
+
* explicit `reportPath` instead.
|
|
87
|
+
*/
|
|
88
|
+
export function writeCostComparisonReport(result, reportPath) {
|
|
89
|
+
const path = resolve(reportPath ?? DEFAULT_REPORT_PATH);
|
|
90
|
+
mkdirSync(dirname(path), { recursive: true });
|
|
91
|
+
writeFileSync(path, `${JSON.stringify(result, null, 2)}\n`, "utf-8");
|
|
92
|
+
}
|
|
93
|
+
//# sourceMappingURL=cost-comparison.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"cost-comparison.js","sourceRoot":"","sources":["../src/cost-comparison.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;;;;;;;;GAyBG;AAEH,OAAO,EAAE,SAAS,EAAE,aAAa,EAAE,MAAM,SAAS,CAAC;AACnD,OAAO,EAAE,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AAC7C,OAAO,EAAE,YAAY,EAAE,WAAW,EAAE,MAAM,eAAe,CAAC;AAiC1D;;;;;;;GAOG;AACH,MAAM,eAAe,GAAG,CAAC,CAAC;AAE1B;;;;;GAKG;AACH,MAAM,sBAAsB,GAAG,GAAG,CAAC;AAEnC,gGAAgG;AAChG,MAAM,mBAAmB,GAAG,wCAAwC,CAAC;AAErE;;;;;;;;;;;;;;GAcG;AACH,MAAM,UAAU,WAAW,CAC1B,YAAoB,EACpB,kBAA0B,EAC1B,KAAoB,EACpB,WAAoB;IAEpB,MAAM,OAAO,GAAG,WAAW,CAAC,WAAW,CAAC,CAAC,MAAM,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,YAAY,KAAK,YAAY,CAAC,CAAC;IAChG,MAAM,UAAU,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,KAAK,EAAE,EAAE,CAAC,GAAG,GAAG,KAAK,CAAC,gBAAgB,EAAE,CAAC,CAAC,CAAC;IACnF,0EAA0E;IAC1E,4EAA4E;IAC5E,4DAA4D;IAC5D,MAAM,QAAQ,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,IAAI,KAAK,UAAU,CAAC,CAAC,MAAM,CAAC;IAE7E,MAAM,oBAAoB,GAAG,IAAI,CAAC,GAAG,CAAC,KAAK,CAAC,aAAa,EAAE,IAAI,CAAC,IAAI,CAAC,kBAAkB,GAAG,eAAe,CAAC,CAAC,CAAC;IAC5G,MAAM,eAAe,GAAG,YAAY,CACnC,KAAK,CAAC,QAAQ,EACd,KAAK,CAAC,SAAS,EACf,oBAAoB,EACpB,sBAAsB,EACtB,CAAC,CACD,CAAC;IAEF,OAAO;QACN,IAAI,EAAE,YAAY;QAClB,UAAU;QACV,eAAe;QACf,QAAQ;QACR,kBAAkB;KAClB,CAAC;AACH,CAAC;AAED;;;;;;;GAOG;AACH,MAAM,UAAU,yBAAyB,CAAC,MAA4B,EAAE,UAAmB;IAC1F,MAAM,IAAI,GAAG,OAAO,CAAC,UAAU,IAAI,mBAAmB,CAAC,CAAC;IACxD,SAAS,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;IAC9C,aAAa,CAAC,IAAI,EAAE,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC,IAAI,EAAE,OAAO,CAAC,CAAC;AACtE,CAAC","sourcesContent":["/**\n * Cost-comparison harness: RLM trajectory cost vs. a truncate-and-single-call\n * baseline for the same task.\n *\n * `rlmCostUsd` is the real, already-incurred cost of a trajectory recovered\n * from `@draht/router`'s cost log (every root + sub call `createRouterBackedSession`\n * logs is tagged with one `trajectoryId` -- see router-session.ts). There is\n * nothing to \"compare\" about that half; it's just a sum-and-filter over\n * already-logged `CostEntry` rows.\n *\n * `baselineCostUsd` is necessarily an *estimate*, not a measurement: it models\n * the cost of the simplest plausible non-RLM alternative for an oversize\n * task -- truncate the context down to whatever fits the target model's\n * context window, then make one single call -- and prices that hypothetical\n * call via `@draht/router`'s `estimateCost`. The specific truncation strategy\n * modeled here is deliberately the crudest one: keep the first N characters\n * of the context that fit the window (a `CHARS_PER_TOKEN` heuristic converts\n * the token-denominated `contextWindow` into a character budget). A smarter\n * baseline (e.g. picking the *most relevant* N characters via embedding\n * search, or summarizing before truncating) would likely produce different\n * numbers -- that's out of scope here. This harness answers \"what would the\n * cheapest naive single-call alternative have cost\", not \"what is the best\n * possible non-RLM approach\". See\n * .planning/phases/30-eval-observability-docs/30-01-PLAN.md, Architecture\n * section 3, and its Risks section.\n */\n\nimport { mkdirSync, writeFileSync } from \"node:fs\";\nimport { dirname, resolve } from \"node:path\";\nimport { estimateCost, readCostLog } from \"@draht/router\";\n\n/** Which target model the truncate-and-single-call baseline is priced against. */\nexport interface BaselineModel {\n\tprovider: string;\n\tmodelName: string;\n\t/** Context window, in tokens -- used to compute how much of the context the baseline call could fit after truncation. */\n\tcontextWindow: number;\n}\n\n/** Result of comparing one RLM trajectory's real cost against a truncate-and-single-call baseline for the same task. */\nexport interface CostComparisonResult {\n\t/**\n\t * Identifies which task/trajectory this comparison covers. There is no\n\t * separate task-name concept tracked elsewhere in this codebase, so this\n\t * is the `trajectoryId` itself -- callers that want a human-readable label\n\t * should track that mapping themselves.\n\t */\n\ttask: string;\n\t/** Sum of every cost-log entry (root call + every `llm_query` sub-call) tagged with this trajectoryId. */\n\trlmCostUsd: number;\n\t/**\n\t * Estimated cost of truncating the context to fit `model`'s window and\n\t * making one single call instead (see the module doc for the exact\n\t * truncation approach modeled).\n\t */\n\tbaselineCostUsd: number;\n\t/** Number of root-LLM steps the RLM trajectory took (one \"rlm-root\"-role cost entry per step -- see router-session.ts's `rootLlm`). */\n\trlmSteps: number;\n\t/** The full (untruncated) context length, in characters, the RLM trajectory operated over. */\n\tcontextLengthChars: number;\n}\n\n/**\n * Rough heuristic: ~4 characters per token for English-ish text. This is the\n * same order-of-magnitude approximation used broadly across LLM tooling when\n * an exact tokenizer isn't wired in for a given model -- it is NOT\n * model-exact, and the `baselineCostUsd` this produces should be read as\n * \"same ballpark\", not \"precise\". See the module doc above and the Risks\n * section of the phase 30 plan.\n */\nconst CHARS_PER_TOKEN = 4;\n\n/**\n * Fixed assumed output length (tokens) for the baseline's single call -- a\n * plausible \"here's the answer\" response size. Deliberately not modeled\n * per-task: this harness needs a reasonable, explicitly-documented constant,\n * not a task-specific prediction of how long an answer would be.\n */\nconst BASELINE_OUTPUT_TOKENS = 500;\n\n/** Default path the \"eval report\" JSON is written to/read from, relative to `process.cwd()`. */\nconst DEFAULT_REPORT_PATH = \".draht/rlm/cost-comparison-report.json\";\n\n/**\n * Compares an already-run RLM trajectory's real cost (read back from the\n * cost log at `costLogPath`, filtered by `trajectoryId`) against the\n * estimated cost of a truncate-and-single-call baseline for the same task:\n * truncate `contextLengthChars` down to whatever fits `model.contextWindow`\n * (first-N-characters, converted via `CHARS_PER_TOKEN`), then price one call\n * to `model` via `@draht/router`'s `estimateCost`.\n *\n * Doesn't run anything itself -- the RLM trajectory must already have run\n * (via `createRouterBackedSession`, logging its cost entries under\n * `costLogPath`) before calling this; the baseline call is hypothetical and\n * never actually dispatched. This function is pure arithmetic over its\n * inputs plus the cost log, and makes no claim about which side is cheaper --\n * that depends entirely on the task.\n */\nexport function compareCost(\n\ttrajectoryId: string,\n\tcontextLengthChars: number,\n\tmodel: BaselineModel,\n\tcostLogPath?: string,\n): CostComparisonResult {\n\tconst entries = readCostLog(costLogPath).filter((entry) => entry.trajectoryId === trajectoryId);\n\tconst rlmCostUsd = entries.reduce((sum, entry) => sum + entry.estimatedCostUsd, 0);\n\t// Exactly one \"rlm-root\"-role cost entry is logged per root-LLM step (see\n\t// router-session.ts's `logRouterCall(\"rlm-root\", ...)` inside `rootLlm`) --\n\t// sub-calls (role \"rlm-sub\") don't count as separate steps.\n\tconst rlmSteps = entries.filter((entry) => entry.role === \"rlm-root\").length;\n\n\tconst truncatedInputTokens = Math.min(model.contextWindow, Math.ceil(contextLengthChars / CHARS_PER_TOKEN));\n\tconst baselineCostUsd = estimateCost(\n\t\tmodel.provider,\n\t\tmodel.modelName,\n\t\ttruncatedInputTokens,\n\t\tBASELINE_OUTPUT_TOKENS,\n\t\t0,\n\t);\n\n\treturn {\n\t\ttask: trajectoryId,\n\t\trlmCostUsd,\n\t\tbaselineCostUsd,\n\t\trlmSteps,\n\t\tcontextLengthChars,\n\t};\n}\n\n/**\n * Writes a `CostComparisonResult` to disk as the \"eval report\" R30-EVAL.3\n * asks for, creating parent directories as needed. Defaults to\n * `.draht/rlm/cost-comparison-report.json` (relative to `process.cwd()`),\n * mirroring `trajectory.ts`'s `DEFAULT_LOG_DIR` convention -- callers that\n * don't want to touch that real location (e.g. tests) should pass an\n * explicit `reportPath` instead.\n */\nexport function writeCostComparisonReport(result: CostComparisonResult, reportPath?: string): void {\n\tconst path = resolve(reportPath ?? DEFAULT_REPORT_PATH);\n\tmkdirSync(dirname(path), { recursive: true });\n\twriteFileSync(path, `${JSON.stringify(result, null, 2)}\\n`, \"utf-8\");\n}\n"]}
|
package/dist/index.d.ts
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
export type { InputSource, LoadedInput } from "./loaders.js";
|
|
2
|
+
export { loadInput, parseInputArg } from "./loaders.js";
|
|
3
|
+
export type { PromptTier, PromptVars } from "./prompts.js";
|
|
4
|
+
export { renderPrompt, selectTier } from "./prompts.js";
|
|
5
|
+
export type { CreateRouterBackedSessionOptions } from "./router-session.js";
|
|
6
|
+
export { createRouterBackedSession } from "./router-session.js";
|
|
7
|
+
export { extractPythonCode, pythonReprToValue, RlmSession, truncateStdout } from "./session.js";
|
|
8
|
+
export type { TrajectoryFinalEntry, TrajectoryStepEntry } from "./trajectory.js";
|
|
9
|
+
export { appendTrajectoryEntry, readTrajectory } from "./trajectory.js";
|
|
10
|
+
export type { RlmHistoryEntry, RlmResult, RlmResultKind, RlmSessionOptions } from "./types.js";
|
|
11
|
+
//# sourceMappingURL=index.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAeA,YAAY,EAAE,WAAW,EAAE,WAAW,EAAE,MAAM,cAAc,CAAC;AAC7D,OAAO,EAAE,SAAS,EAAE,aAAa,EAAE,MAAM,cAAc,CAAC;AACxD,YAAY,EAAE,UAAU,EAAE,UAAU,EAAE,MAAM,cAAc,CAAC;AAC3D,OAAO,EAAE,YAAY,EAAE,UAAU,EAAE,MAAM,cAAc,CAAC;AACxD,YAAY,EAAE,gCAAgC,EAAE,MAAM,qBAAqB,CAAC;AAC5E,OAAO,EAAE,yBAAyB,EAAE,MAAM,qBAAqB,CAAC;AAChE,OAAO,EAAE,iBAAiB,EAAE,iBAAiB,EAAE,UAAU,EAAE,cAAc,EAAE,MAAM,cAAc,CAAC;AAMhG,YAAY,EAAE,oBAAoB,EAAE,mBAAmB,EAAE,MAAM,iBAAiB,CAAC;AACjF,OAAO,EAAE,qBAAqB,EAAE,cAAc,EAAE,MAAM,iBAAiB,CAAC;AACxE,YAAY,EAAE,eAAe,EAAE,SAAS,EAAE,aAAa,EAAE,iBAAiB,EAAE,MAAM,YAAY,CAAC"}
|
package/dist/index.js
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
// @draht/rlm — Recursive Language Models core primitives.
|
|
2
|
+
//
|
|
3
|
+
// The root loop (`RlmSession`) and its Python REPL driver
|
|
4
|
+
// (`python/repl_driver.py`) implement the inference-time scaffold from
|
|
5
|
+
// Zhang, Kraska, Khattab (2026), *Recursive Language Models*
|
|
6
|
+
// (arXiv:2512.24601). Real `@draht/router` wiring for `llmQuery`/the root
|
|
7
|
+
// LLM, sandboxing, and CLI integration land in later phases (27-29); see
|
|
8
|
+
// .planning/phases/26-rlm-core-primitives/26-01-PLAN.md.
|
|
9
|
+
export { loadInput, parseInputArg } from "./loaders.js";
|
|
10
|
+
export { renderPrompt, selectTier } from "./prompts.js";
|
|
11
|
+
export { createRouterBackedSession } from "./router-session.js";
|
|
12
|
+
export { extractPythonCode, pythonReprToValue, RlmSession, truncateStdout } from "./session.js";
|
|
13
|
+
export { appendTrajectoryEntry, readTrajectory } from "./trajectory.js";
|
|
14
|
+
//# sourceMappingURL=index.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,0DAA0D;AAC1D,EAAE;AACF,0DAA0D;AAC1D,uEAAuE;AACvE,6DAA6D;AAC7D,0EAA0E;AAC1E,yEAAyE;AACzE,yDAAyD;AASzD,OAAO,EAAE,SAAS,EAAE,aAAa,EAAE,MAAM,cAAc,CAAC;AAExD,OAAO,EAAE,YAAY,EAAE,UAAU,EAAE,MAAM,cAAc,CAAC;AAExD,OAAO,EAAE,yBAAyB,EAAE,MAAM,qBAAqB,CAAC;AAChE,OAAO,EAAE,iBAAiB,EAAE,iBAAiB,EAAE,UAAU,EAAE,cAAc,EAAE,MAAM,cAAc,CAAC;AAOhG,OAAO,EAAE,qBAAqB,EAAE,cAAc,EAAE,MAAM,iBAAiB,CAAC","sourcesContent":["// @draht/rlm — Recursive Language Models core primitives.\n//\n// The root loop (`RlmSession`) and its Python REPL driver\n// (`python/repl_driver.py`) implement the inference-time scaffold from\n// Zhang, Kraska, Khattab (2026), *Recursive Language Models*\n// (arXiv:2512.24601). Real `@draht/router` wiring for `llmQuery`/the root\n// LLM, sandboxing, and CLI integration land in later phases (27-29); see\n// .planning/phases/26-rlm-core-primitives/26-01-PLAN.md.\n\n// Input loaders -- turn a raw `--input`/`/rlm <input>` argument (file, glob,\n// URL, or `knowledge:<client-slug>`) into loaded prompt content. See\n// .planning/phases/29-agent-cli-integration/29-01-PLAN.md, Architecture\n// section 1. Re-exported from the package root so consumers outside this\n// package (e.g. `@draht/rlm-agent`, the `draht rlm` CLI) can import them via\n// the public `@draht/rlm` entry point instead of reaching into `src/`.\nexport type { InputSource, LoadedInput } from \"./loaders.js\";\nexport { loadInput, parseInputArg } from \"./loaders.js\";\nexport type { PromptTier, PromptVars } from \"./prompts.js\";\nexport { renderPrompt, selectTier } from \"./prompts.js\";\nexport type { CreateRouterBackedSessionOptions } from \"./router-session.js\";\nexport { createRouterBackedSession } from \"./router-session.js\";\nexport { extractPythonCode, pythonReprToValue, RlmSession, truncateStdout } from \"./session.js\";\n// Trajectory JSONL logging/replay (Phase 30) -- see\n// .planning/phases/30-eval-observability-docs/30-01-PLAN.md, Architecture\n// section 1. Re-exported from the package root so consumers outside this\n// package (e.g. `draht rlm replay`) can import via the public `@draht/rlm`\n// entry point instead of reaching into `src/`.\nexport type { TrajectoryFinalEntry, TrajectoryStepEntry } from \"./trajectory.js\";\nexport { appendTrajectoryEntry, readTrajectory } from \"./trajectory.js\";\nexport type { RlmHistoryEntry, RlmResult, RlmResultKind, RlmSessionOptions } from \"./types.js\";\n"]}
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Input loaders -- turn a raw `--input`/`/rlm <input>` argument into loaded
|
|
3
|
+
* text content ready to become an `RlmSession`'s `prompt`. Covers four kinds
|
|
4
|
+
* of source: a single file, a glob pattern, an `http(s)://` URL, and a named
|
|
5
|
+
* client knowledge base (via `@draht/knowledge`). See
|
|
6
|
+
* .planning/phases/29-agent-cli-integration/29-01-PLAN.md, Architecture
|
|
7
|
+
* section 1.
|
|
8
|
+
*
|
|
9
|
+
* `parseInputArg` classifies the raw string; `loadInput` does the actual
|
|
10
|
+
* I/O. Kept as two functions so callers (CLI, `/rlm` command, `rlm_query`
|
|
11
|
+
* tool) can validate/echo back what kind of source they resolved before
|
|
12
|
+
* paying the cost of loading it.
|
|
13
|
+
*/
|
|
14
|
+
export type InputSource = {
|
|
15
|
+
type: "file";
|
|
16
|
+
path: string;
|
|
17
|
+
} | {
|
|
18
|
+
type: "glob";
|
|
19
|
+
pattern: string;
|
|
20
|
+
cwd: string;
|
|
21
|
+
} | {
|
|
22
|
+
type: "url";
|
|
23
|
+
url: string;
|
|
24
|
+
} | {
|
|
25
|
+
type: "knowledge";
|
|
26
|
+
clientSlug: string;
|
|
27
|
+
};
|
|
28
|
+
export interface LoadedInput {
|
|
29
|
+
/** The loaded/concatenated text content. */
|
|
30
|
+
content: string;
|
|
31
|
+
/** What kind of content `content` holds (e.g. "text", "code", "logs", "json") -- see `PromptVars.contextType`. */
|
|
32
|
+
contextType: string;
|
|
33
|
+
}
|
|
34
|
+
/**
|
|
35
|
+
* Classifies a raw `--input`/`/rlm` argument into an {@link InputSource}:
|
|
36
|
+
* - `knowledge:<client-slug>` -> `{ type: "knowledge", clientSlug }`.
|
|
37
|
+
* - `http://...` / `https://...` -> `{ type: "url", url }`.
|
|
38
|
+
* - anything containing glob metacharacters (`* ? { } [ ] !`) -> `{ type: "glob", pattern, cwd }`.
|
|
39
|
+
* - everything else -> `{ type: "file", path }`, resolved against `cwd` if relative.
|
|
40
|
+
*/
|
|
41
|
+
export declare function parseInputArg(input: string, cwd: string): InputSource;
|
|
42
|
+
/** Loads the given {@link InputSource}, returning its content and the context type to advertise to the RLM root loop. */
|
|
43
|
+
export declare function loadInput(source: InputSource): Promise<LoadedInput>;
|
|
44
|
+
//# sourceMappingURL=loaders.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"loaders.d.ts","sourceRoot":"","sources":["../src/loaders.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;GAYG;AAMH,MAAM,MAAM,WAAW,GACpB;IAAE,IAAI,EAAE,MAAM,CAAC;IAAC,IAAI,EAAE,MAAM,CAAA;CAAE,GAC9B;IAAE,IAAI,EAAE,MAAM,CAAC;IAAC,OAAO,EAAE,MAAM,CAAC;IAAC,GAAG,EAAE,MAAM,CAAA;CAAE,GAC9C;IAAE,IAAI,EAAE,KAAK,CAAC;IAAC,GAAG,EAAE,MAAM,CAAA;CAAE,GAC5B;IAAE,IAAI,EAAE,WAAW,CAAC;IAAC,UAAU,EAAE,MAAM,CAAA;CAAE,CAAC;AAE7C,MAAM,WAAW,WAAW;IAC3B,4CAA4C;IAC5C,OAAO,EAAE,MAAM,CAAC;IAChB,kHAAkH;IAClH,WAAW,EAAE,MAAM,CAAC;CACpB;AAkCD;;;;;;GAMG;AACH,wBAAgB,aAAa,CAAC,KAAK,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,GAAG,WAAW,CAiBrE;AAED,yHAAyH;AACzH,wBAAsB,SAAS,CAAC,MAAM,EAAE,WAAW,GAAG,OAAO,CAAC,WAAW,CAAC,CAgBzE"}
|
package/dist/loaders.js
ADDED
|
@@ -0,0 +1,156 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Input loaders -- turn a raw `--input`/`/rlm <input>` argument into loaded
|
|
3
|
+
* text content ready to become an `RlmSession`'s `prompt`. Covers four kinds
|
|
4
|
+
* of source: a single file, a glob pattern, an `http(s)://` URL, and a named
|
|
5
|
+
* client knowledge base (via `@draht/knowledge`). See
|
|
6
|
+
* .planning/phases/29-agent-cli-integration/29-01-PLAN.md, Architecture
|
|
7
|
+
* section 1.
|
|
8
|
+
*
|
|
9
|
+
* `parseInputArg` classifies the raw string; `loadInput` does the actual
|
|
10
|
+
* I/O. Kept as two functions so callers (CLI, `/rlm` command, `rlm_query`
|
|
11
|
+
* tool) can validate/echo back what kind of source they resolved before
|
|
12
|
+
* paying the cost of loading it.
|
|
13
|
+
*/
|
|
14
|
+
import { glob, readFile } from "node:fs/promises";
|
|
15
|
+
import * as path from "node:path";
|
|
16
|
+
/**
|
|
17
|
+
* Recognized prefix for the knowledge-base input syntax: `knowledge:<client-slug>`
|
|
18
|
+
* (e.g. `knowledge:acme-corp`), where `<client-slug>` matches `ClientConfig.slug`
|
|
19
|
+
* from `@draht/knowledge`'s `client-config.ts`.
|
|
20
|
+
*/
|
|
21
|
+
const KNOWLEDGE_PREFIX = "knowledge:";
|
|
22
|
+
/** Characters that mark an `--input` string as a glob pattern rather than a literal file path. */
|
|
23
|
+
const GLOB_CHARS = /[*?{}[\]!]/;
|
|
24
|
+
const CODE_EXTENSIONS = new Set([
|
|
25
|
+
".ts",
|
|
26
|
+
".tsx",
|
|
27
|
+
".js",
|
|
28
|
+
".jsx",
|
|
29
|
+
".mjs",
|
|
30
|
+
".cjs",
|
|
31
|
+
".py",
|
|
32
|
+
".go",
|
|
33
|
+
".rs",
|
|
34
|
+
".java",
|
|
35
|
+
".c",
|
|
36
|
+
".cc",
|
|
37
|
+
".cpp",
|
|
38
|
+
".h",
|
|
39
|
+
".hpp",
|
|
40
|
+
".rb",
|
|
41
|
+
".sh",
|
|
42
|
+
".swift",
|
|
43
|
+
".kt",
|
|
44
|
+
]);
|
|
45
|
+
/**
|
|
46
|
+
* Classifies a raw `--input`/`/rlm` argument into an {@link InputSource}:
|
|
47
|
+
* - `knowledge:<client-slug>` -> `{ type: "knowledge", clientSlug }`.
|
|
48
|
+
* - `http://...` / `https://...` -> `{ type: "url", url }`.
|
|
49
|
+
* - anything containing glob metacharacters (`* ? { } [ ] !`) -> `{ type: "glob", pattern, cwd }`.
|
|
50
|
+
* - everything else -> `{ type: "file", path }`, resolved against `cwd` if relative.
|
|
51
|
+
*/
|
|
52
|
+
export function parseInputArg(input, cwd) {
|
|
53
|
+
const trimmed = input.trim();
|
|
54
|
+
if (trimmed.startsWith(KNOWLEDGE_PREFIX)) {
|
|
55
|
+
const clientSlug = trimmed.slice(KNOWLEDGE_PREFIX.length).trim();
|
|
56
|
+
return { type: "knowledge", clientSlug };
|
|
57
|
+
}
|
|
58
|
+
if (/^https?:\/\//i.test(trimmed)) {
|
|
59
|
+
return { type: "url", url: trimmed };
|
|
60
|
+
}
|
|
61
|
+
if (GLOB_CHARS.test(trimmed)) {
|
|
62
|
+
return { type: "glob", pattern: trimmed, cwd };
|
|
63
|
+
}
|
|
64
|
+
return { type: "file", path: path.isAbsolute(trimmed) ? trimmed : path.join(cwd, trimmed) };
|
|
65
|
+
}
|
|
66
|
+
/** Loads the given {@link InputSource}, returning its content and the context type to advertise to the RLM root loop. */
|
|
67
|
+
export async function loadInput(source) {
|
|
68
|
+
switch (source.type) {
|
|
69
|
+
case "file":
|
|
70
|
+
return loadFile(source.path);
|
|
71
|
+
case "glob":
|
|
72
|
+
return loadGlob(source.pattern, source.cwd);
|
|
73
|
+
case "url":
|
|
74
|
+
return loadUrl(source.url);
|
|
75
|
+
case "knowledge":
|
|
76
|
+
return loadKnowledge(source.clientSlug);
|
|
77
|
+
default: {
|
|
78
|
+
// Exhaustiveness check -- widen InputSource, forget a branch, get a compile error here.
|
|
79
|
+
const _never = source;
|
|
80
|
+
throw new Error(`loadInput: unknown source type "${_never.type}"`);
|
|
81
|
+
}
|
|
82
|
+
}
|
|
83
|
+
}
|
|
84
|
+
async function loadFile(filePath) {
|
|
85
|
+
const content = await readFile(filePath, "utf-8");
|
|
86
|
+
return { content, contextType: inferContextType(filePath) };
|
|
87
|
+
}
|
|
88
|
+
async function loadGlob(pattern, cwd) {
|
|
89
|
+
const matches = [];
|
|
90
|
+
for await (const match of glob(pattern, { cwd })) {
|
|
91
|
+
matches.push(match);
|
|
92
|
+
}
|
|
93
|
+
matches.sort();
|
|
94
|
+
if (matches.length === 0) {
|
|
95
|
+
throw new Error(`loadInput: glob pattern "${pattern}" matched no files (cwd: ${cwd})`);
|
|
96
|
+
}
|
|
97
|
+
const contextTypes = new Set();
|
|
98
|
+
const parts = await Promise.all(matches.map(async (relPath) => {
|
|
99
|
+
const absPath = path.isAbsolute(relPath) ? relPath : path.join(cwd, relPath);
|
|
100
|
+
const content = await readFile(absPath, "utf-8");
|
|
101
|
+
contextTypes.add(inferContextType(absPath));
|
|
102
|
+
return `--- source: ${absPath} ---\n${content}`;
|
|
103
|
+
}));
|
|
104
|
+
// A single matched file's context type carries through; a mixed match falls back to "text".
|
|
105
|
+
const contextType = contextTypes.size === 1 ? [...contextTypes][0] : "text";
|
|
106
|
+
return { content: parts.join("\n\n"), contextType };
|
|
107
|
+
}
|
|
108
|
+
async function loadUrl(url) {
|
|
109
|
+
const response = await fetch(url);
|
|
110
|
+
if (!response.ok) {
|
|
111
|
+
throw new Error(`loadInput: fetching "${url}" failed with status ${response.status}`);
|
|
112
|
+
}
|
|
113
|
+
const content = await response.text();
|
|
114
|
+
const contentType = response.headers.get("content-type") ?? "";
|
|
115
|
+
return { content, contextType: contentType.includes("json") ? "json" : "text" };
|
|
116
|
+
}
|
|
117
|
+
/** Chunks per section pulled from the client's knowledge store -- generous enough to cover most real stores. */
|
|
118
|
+
const KNOWLEDGE_SEARCH_LIMIT = 20;
|
|
119
|
+
async function loadKnowledge(clientSlug) {
|
|
120
|
+
const { KnowledgeManager } = await import("@draht/knowledge");
|
|
121
|
+
const manager = new KnowledgeManager(clientSlug);
|
|
122
|
+
try {
|
|
123
|
+
const [conventions, decisions] = await Promise.all([
|
|
124
|
+
manager.searchKnowledge("project conventions and AGENTS.md guidance", "general", KNOWLEDGE_SEARCH_LIMIT),
|
|
125
|
+
manager.searchKnowledge("past decisions and rationale", "decide", KNOWLEDGE_SEARCH_LIMIT),
|
|
126
|
+
]);
|
|
127
|
+
const sections = [
|
|
128
|
+
formatKnowledgeSection("AGENTS.md / conventions", conventions),
|
|
129
|
+
formatKnowledgeSection("Past decisions", decisions),
|
|
130
|
+
].filter((section) => section !== null);
|
|
131
|
+
if (sections.length === 0) {
|
|
132
|
+
throw new Error(`loadInput: no knowledge found for client "${clientSlug}"`);
|
|
133
|
+
}
|
|
134
|
+
return { content: sections.join("\n\n"), contextType: "text" };
|
|
135
|
+
}
|
|
136
|
+
finally {
|
|
137
|
+
manager.close();
|
|
138
|
+
}
|
|
139
|
+
}
|
|
140
|
+
function formatKnowledgeSection(title, results) {
|
|
141
|
+
if (results.length === 0)
|
|
142
|
+
return null;
|
|
143
|
+
const body = results.map((r) => r.content).join("\n\n");
|
|
144
|
+
return `## ${title}\n\n${body}`;
|
|
145
|
+
}
|
|
146
|
+
function inferContextType(filePath) {
|
|
147
|
+
const ext = path.extname(filePath).toLowerCase();
|
|
148
|
+
if (ext === ".json")
|
|
149
|
+
return "json";
|
|
150
|
+
if (ext === ".log")
|
|
151
|
+
return "logs";
|
|
152
|
+
if (CODE_EXTENSIONS.has(ext))
|
|
153
|
+
return "code";
|
|
154
|
+
return "text";
|
|
155
|
+
}
|
|
156
|
+
//# sourceMappingURL=loaders.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"loaders.js","sourceRoot":"","sources":["../src/loaders.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;GAYG;AAEH,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,MAAM,kBAAkB,CAAC;AAClD,OAAO,KAAK,IAAI,MAAM,WAAW,CAAC;AAgBlC;;;;GAIG;AACH,MAAM,gBAAgB,GAAG,YAAY,CAAC;AAEtC,kGAAkG;AAClG,MAAM,UAAU,GAAG,YAAY,CAAC;AAEhC,MAAM,eAAe,GAAG,IAAI,GAAG,CAAC;IAC/B,KAAK;IACL,MAAM;IACN,KAAK;IACL,MAAM;IACN,MAAM;IACN,MAAM;IACN,KAAK;IACL,KAAK;IACL,KAAK;IACL,OAAO;IACP,IAAI;IACJ,KAAK;IACL,MAAM;IACN,IAAI;IACJ,MAAM;IACN,KAAK;IACL,KAAK;IACL,QAAQ;IACR,KAAK;CACL,CAAC,CAAC;AAEH;;;;;;GAMG;AACH,MAAM,UAAU,aAAa,CAAC,KAAa,EAAE,GAAW;IACvD,MAAM,OAAO,GAAG,KAAK,CAAC,IAAI,EAAE,CAAC;IAE7B,IAAI,OAAO,CAAC,UAAU,CAAC,gBAAgB,CAAC,EAAE,CAAC;QAC1C,MAAM,UAAU,GAAG,OAAO,CAAC,KAAK,CAAC,gBAAgB,CAAC,MAAM,CAAC,CAAC,IAAI,EAAE,CAAC;QACjE,OAAO,EAAE,IAAI,EAAE,WAAW,EAAE,UAAU,EAAE,CAAC;IAC1C,CAAC;IAED,IAAI,eAAe,CAAC,IAAI,CAAC,OAAO,CAAC,EAAE,CAAC;QACnC,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,GAAG,EAAE,OAAO,EAAE,CAAC;IACtC,CAAC;IAED,IAAI,UAAU,CAAC,IAAI,CAAC,OAAO,CAAC,EAAE,CAAC;QAC9B,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,OAAO,EAAE,GAAG,EAAE,CAAC;IAChD,CAAC;IAED,OAAO,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,IAAI,CAAC,UAAU,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,EAAE,OAAO,CAAC,EAAE,CAAC;AAC7F,CAAC;AAED,yHAAyH;AACzH,MAAM,CAAC,KAAK,UAAU,SAAS,CAAC,MAAmB;IAClD,QAAQ,MAAM,CAAC,IAAI,EAAE,CAAC;QACrB,KAAK,MAAM;YACV,OAAO,QAAQ,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;QAC9B,KAAK,MAAM;YACV,OAAO,QAAQ,CAAC,MAAM,CAAC,OAAO,EAAE,MAAM,CAAC,GAAG,CAAC,CAAC;QAC7C,KAAK,KAAK;YACT,OAAO,OAAO,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;QAC5B,KAAK,WAAW;YACf,OAAO,aAAa,CAAC,MAAM,CAAC,UAAU,CAAC,CAAC;QACzC,SAAS,CAAC;YACT,wFAAwF;YACxF,MAAM,MAAM,GAAU,MAAM,CAAC;YAC7B,MAAM,IAAI,KAAK,CAAC,mCAAoC,MAAsB,CAAC,IAAI,GAAG,CAAC,CAAC;QACrF,CAAC;IACF,CAAC;AACF,CAAC;AAED,KAAK,UAAU,QAAQ,CAAC,QAAgB;IACvC,MAAM,OAAO,GAAG,MAAM,QAAQ,CAAC,QAAQ,EAAE,OAAO,CAAC,CAAC;IAClD,OAAO,EAAE,OAAO,EAAE,WAAW,EAAE,gBAAgB,CAAC,QAAQ,CAAC,EAAE,CAAC;AAC7D,CAAC;AAED,KAAK,UAAU,QAAQ,CAAC,OAAe,EAAE,GAAW;IACnD,MAAM,OAAO,GAAa,EAAE,CAAC;IAC7B,IAAI,KAAK,EAAE,MAAM,KAAK,IAAI,IAAI,CAAC,OAAO,EAAE,EAAE,GAAG,EAAE,CAAC,EAAE,CAAC;QAClD,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;IACrB,CAAC;IACD,OAAO,CAAC,IAAI,EAAE,CAAC;IAEf,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC1B,MAAM,IAAI,KAAK,CAAC,4BAA4B,OAAO,4BAA4B,GAAG,GAAG,CAAC,CAAC;IACxF,CAAC;IAED,MAAM,YAAY,GAAG,IAAI,GAAG,EAAU,CAAC;IACvC,MAAM,KAAK,GAAG,MAAM,OAAO,CAAC,GAAG,CAC9B,OAAO,CAAC,GAAG,CAAC,KAAK,EAAE,OAAO,EAAE,EAAE;QAC7B,MAAM,OAAO,GAAG,IAAI,CAAC,UAAU,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,EAAE,OAAO,CAAC,CAAC;QAC7E,MAAM,OAAO,GAAG,MAAM,QAAQ,CAAC,OAAO,EAAE,OAAO,CAAC,CAAC;QACjD,YAAY,CAAC,GAAG,CAAC,gBAAgB,CAAC,OAAO,CAAC,CAAC,CAAC;QAC5C,OAAO,eAAe,OAAO,SAAS,OAAO,EAAE,CAAC;IACjD,CAAC,CAAC,CACF,CAAC;IAEF,4FAA4F;IAC5F,MAAM,WAAW,GAAG,YAAY,CAAC,IAAI,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,GAAG,YAAY,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC;IAC5E,OAAO,EAAE,OAAO,EAAE,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,WAAW,EAAE,CAAC;AACrD,CAAC;AAED,KAAK,UAAU,OAAO,CAAC,GAAW;IACjC,MAAM,QAAQ,GAAG,MAAM,KAAK,CAAC,GAAG,CAAC,CAAC;IAClC,IAAI,CAAC,QAAQ,CAAC,EAAE,EAAE,CAAC;QAClB,MAAM,IAAI,KAAK,CAAC,wBAAwB,GAAG,wBAAwB,QAAQ,CAAC,MAAM,EAAE,CAAC,CAAC;IACvF,CAAC;IACD,MAAM,OAAO,GAAG,MAAM,QAAQ,CAAC,IAAI,EAAE,CAAC;IACtC,MAAM,WAAW,GAAG,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC,cAAc,CAAC,IAAI,EAAE,CAAC;IAC/D,OAAO,EAAE,OAAO,EAAE,WAAW,EAAE,WAAW,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC;AACjF,CAAC;AAED,gHAAgH;AAChH,MAAM,sBAAsB,GAAG,EAAE,CAAC;AAElC,KAAK,UAAU,aAAa,CAAC,UAAkB;IAC9C,MAAM,EAAE,gBAAgB,EAAE,GAAG,MAAM,MAAM,CAAC,kBAAkB,CAAC,CAAC;IAC9D,MAAM,OAAO,GAAG,IAAI,gBAAgB,CAAC,UAAU,CAAC,CAAC;IACjD,IAAI,CAAC;QACJ,MAAM,CAAC,WAAW,EAAE,SAAS,CAAC,GAAG,MAAM,OAAO,CAAC,GAAG,CAAC;YAClD,OAAO,CAAC,eAAe,CAAC,4CAA4C,EAAE,SAAS,EAAE,sBAAsB,CAAC;YACxG,OAAO,CAAC,eAAe,CAAC,8BAA8B,EAAE,QAAQ,EAAE,sBAAsB,CAAC;SACzF,CAAC,CAAC;QAEH,MAAM,QAAQ,GAAG;YAChB,sBAAsB,CAAC,yBAAyB,EAAE,WAAW,CAAC;YAC9D,sBAAsB,CAAC,gBAAgB,EAAE,SAAS,CAAC;SACnD,CAAC,MAAM,CAAC,CAAC,OAAO,EAAqB,EAAE,CAAC,OAAO,KAAK,IAAI,CAAC,CAAC;QAE3D,IAAI,QAAQ,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YAC3B,MAAM,IAAI,KAAK,CAAC,6CAA6C,UAAU,GAAG,CAAC,CAAC;QAC7E,CAAC;QAED,OAAO,EAAE,OAAO,EAAE,QAAQ,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,WAAW,EAAE,MAAM,EAAE,CAAC;IAChE,CAAC;YAAS,CAAC;QACV,OAAO,CAAC,KAAK,EAAE,CAAC;IACjB,CAAC;AACF,CAAC;AAED,SAAS,sBAAsB,CAAC,KAAa,EAAE,OAAuB;IACrE,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,IAAI,CAAC;IACtC,MAAM,IAAI,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;IACxD,OAAO,MAAM,KAAK,OAAO,IAAI,EAAE,CAAC;AACjC,CAAC;AAED,SAAS,gBAAgB,CAAC,QAAgB;IACzC,MAAM,GAAG,GAAG,IAAI,CAAC,OAAO,CAAC,QAAQ,CAAC,CAAC,WAAW,EAAE,CAAC;IACjD,IAAI,GAAG,KAAK,OAAO;QAAE,OAAO,MAAM,CAAC;IACnC,IAAI,GAAG,KAAK,MAAM;QAAE,OAAO,MAAM,CAAC;IAClC,IAAI,eAAe,CAAC,GAAG,CAAC,GAAG,CAAC;QAAE,OAAO,MAAM,CAAC;IAC5C,OAAO,MAAM,CAAC;AACf,CAAC","sourcesContent":["/**\n * Input loaders -- turn a raw `--input`/`/rlm <input>` argument into loaded\n * text content ready to become an `RlmSession`'s `prompt`. Covers four kinds\n * of source: a single file, a glob pattern, an `http(s)://` URL, and a named\n * client knowledge base (via `@draht/knowledge`). See\n * .planning/phases/29-agent-cli-integration/29-01-PLAN.md, Architecture\n * section 1.\n *\n * `parseInputArg` classifies the raw string; `loadInput` does the actual\n * I/O. Kept as two functions so callers (CLI, `/rlm` command, `rlm_query`\n * tool) can validate/echo back what kind of source they resolved before\n * paying the cost of loading it.\n */\n\nimport { glob, readFile } from \"node:fs/promises\";\nimport * as path from \"node:path\";\nimport type { SearchResult } from \"@draht/knowledge\";\n\nexport type InputSource =\n\t| { type: \"file\"; path: string }\n\t| { type: \"glob\"; pattern: string; cwd: string }\n\t| { type: \"url\"; url: string }\n\t| { type: \"knowledge\"; clientSlug: string };\n\nexport interface LoadedInput {\n\t/** The loaded/concatenated text content. */\n\tcontent: string;\n\t/** What kind of content `content` holds (e.g. \"text\", \"code\", \"logs\", \"json\") -- see `PromptVars.contextType`. */\n\tcontextType: string;\n}\n\n/**\n * Recognized prefix for the knowledge-base input syntax: `knowledge:<client-slug>`\n * (e.g. `knowledge:acme-corp`), where `<client-slug>` matches `ClientConfig.slug`\n * from `@draht/knowledge`'s `client-config.ts`.\n */\nconst KNOWLEDGE_PREFIX = \"knowledge:\";\n\n/** Characters that mark an `--input` string as a glob pattern rather than a literal file path. */\nconst GLOB_CHARS = /[*?{}[\\]!]/;\n\nconst CODE_EXTENSIONS = new Set([\n\t\".ts\",\n\t\".tsx\",\n\t\".js\",\n\t\".jsx\",\n\t\".mjs\",\n\t\".cjs\",\n\t\".py\",\n\t\".go\",\n\t\".rs\",\n\t\".java\",\n\t\".c\",\n\t\".cc\",\n\t\".cpp\",\n\t\".h\",\n\t\".hpp\",\n\t\".rb\",\n\t\".sh\",\n\t\".swift\",\n\t\".kt\",\n]);\n\n/**\n * Classifies a raw `--input`/`/rlm` argument into an {@link InputSource}:\n * - `knowledge:<client-slug>` -> `{ type: \"knowledge\", clientSlug }`.\n * - `http://...` / `https://...` -> `{ type: \"url\", url }`.\n * - anything containing glob metacharacters (`* ? { } [ ] !`) -> `{ type: \"glob\", pattern, cwd }`.\n * - everything else -> `{ type: \"file\", path }`, resolved against `cwd` if relative.\n */\nexport function parseInputArg(input: string, cwd: string): InputSource {\n\tconst trimmed = input.trim();\n\n\tif (trimmed.startsWith(KNOWLEDGE_PREFIX)) {\n\t\tconst clientSlug = trimmed.slice(KNOWLEDGE_PREFIX.length).trim();\n\t\treturn { type: \"knowledge\", clientSlug };\n\t}\n\n\tif (/^https?:\\/\\//i.test(trimmed)) {\n\t\treturn { type: \"url\", url: trimmed };\n\t}\n\n\tif (GLOB_CHARS.test(trimmed)) {\n\t\treturn { type: \"glob\", pattern: trimmed, cwd };\n\t}\n\n\treturn { type: \"file\", path: path.isAbsolute(trimmed) ? trimmed : path.join(cwd, trimmed) };\n}\n\n/** Loads the given {@link InputSource}, returning its content and the context type to advertise to the RLM root loop. */\nexport async function loadInput(source: InputSource): Promise<LoadedInput> {\n\tswitch (source.type) {\n\t\tcase \"file\":\n\t\t\treturn loadFile(source.path);\n\t\tcase \"glob\":\n\t\t\treturn loadGlob(source.pattern, source.cwd);\n\t\tcase \"url\":\n\t\t\treturn loadUrl(source.url);\n\t\tcase \"knowledge\":\n\t\t\treturn loadKnowledge(source.clientSlug);\n\t\tdefault: {\n\t\t\t// Exhaustiveness check -- widen InputSource, forget a branch, get a compile error here.\n\t\t\tconst _never: never = source;\n\t\t\tthrow new Error(`loadInput: unknown source type \"${(_never as InputSource).type}\"`);\n\t\t}\n\t}\n}\n\nasync function loadFile(filePath: string): Promise<LoadedInput> {\n\tconst content = await readFile(filePath, \"utf-8\");\n\treturn { content, contextType: inferContextType(filePath) };\n}\n\nasync function loadGlob(pattern: string, cwd: string): Promise<LoadedInput> {\n\tconst matches: string[] = [];\n\tfor await (const match of glob(pattern, { cwd })) {\n\t\tmatches.push(match);\n\t}\n\tmatches.sort();\n\n\tif (matches.length === 0) {\n\t\tthrow new Error(`loadInput: glob pattern \"${pattern}\" matched no files (cwd: ${cwd})`);\n\t}\n\n\tconst contextTypes = new Set<string>();\n\tconst parts = await Promise.all(\n\t\tmatches.map(async (relPath) => {\n\t\t\tconst absPath = path.isAbsolute(relPath) ? relPath : path.join(cwd, relPath);\n\t\t\tconst content = await readFile(absPath, \"utf-8\");\n\t\t\tcontextTypes.add(inferContextType(absPath));\n\t\t\treturn `--- source: ${absPath} ---\\n${content}`;\n\t\t}),\n\t);\n\n\t// A single matched file's context type carries through; a mixed match falls back to \"text\".\n\tconst contextType = contextTypes.size === 1 ? [...contextTypes][0] : \"text\";\n\treturn { content: parts.join(\"\\n\\n\"), contextType };\n}\n\nasync function loadUrl(url: string): Promise<LoadedInput> {\n\tconst response = await fetch(url);\n\tif (!response.ok) {\n\t\tthrow new Error(`loadInput: fetching \"${url}\" failed with status ${response.status}`);\n\t}\n\tconst content = await response.text();\n\tconst contentType = response.headers.get(\"content-type\") ?? \"\";\n\treturn { content, contextType: contentType.includes(\"json\") ? \"json\" : \"text\" };\n}\n\n/** Chunks per section pulled from the client's knowledge store -- generous enough to cover most real stores. */\nconst KNOWLEDGE_SEARCH_LIMIT = 20;\n\nasync function loadKnowledge(clientSlug: string): Promise<LoadedInput> {\n\tconst { KnowledgeManager } = await import(\"@draht/knowledge\");\n\tconst manager = new KnowledgeManager(clientSlug);\n\ttry {\n\t\tconst [conventions, decisions] = await Promise.all([\n\t\t\tmanager.searchKnowledge(\"project conventions and AGENTS.md guidance\", \"general\", KNOWLEDGE_SEARCH_LIMIT),\n\t\t\tmanager.searchKnowledge(\"past decisions and rationale\", \"decide\", KNOWLEDGE_SEARCH_LIMIT),\n\t\t]);\n\n\t\tconst sections = [\n\t\t\tformatKnowledgeSection(\"AGENTS.md / conventions\", conventions),\n\t\t\tformatKnowledgeSection(\"Past decisions\", decisions),\n\t\t].filter((section): section is string => section !== null);\n\n\t\tif (sections.length === 0) {\n\t\t\tthrow new Error(`loadInput: no knowledge found for client \"${clientSlug}\"`);\n\t\t}\n\n\t\treturn { content: sections.join(\"\\n\\n\"), contextType: \"text\" };\n\t} finally {\n\t\tmanager.close();\n\t}\n}\n\nfunction formatKnowledgeSection(title: string, results: SearchResult[]): string | null {\n\tif (results.length === 0) return null;\n\tconst body = results.map((r) => r.content).join(\"\\n\\n\");\n\treturn `## ${title}\\n\\n${body}`;\n}\n\nfunction inferContextType(filePath: string): string {\n\tconst ext = path.extname(filePath).toLowerCase();\n\tif (ext === \".json\") return \"json\";\n\tif (ext === \".log\") return \"logs\";\n\tif (CODE_EXTENSIONS.has(ext)) return \"code\";\n\treturn \"text\";\n}\n"]}
|