llm-cost-estimation 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Riddim Software
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md ADDED
@@ -0,0 +1,149 @@
1
+ # llm-cost-estimation
2
+
3
+ Forecast LLM cost for a future issue from historical usage telemetry and issue-size estimates.
4
+
5
+ `llm-cost-estimation` is the pre-work sibling to [`llm-cost-attribution`](../llm-cost-attribution).
6
+ Attribution reports what was spent after work completes.
7
+ This package forecasts what is likely to be spent before work starts.
8
+
9
+ ## What it does
10
+
11
+ It looks at what past issues of a given size *actually* cost and forecasts the same for a new one. Concretely:
12
+
13
+ - Reads **usage records** — one row of cost data per agent **turn** (a turn is one agent request → response) — that follow the [Symphony Cost Telemetry Extension](../specs/symphony-cost-telemetry-extension/SPEC.md).
14
+ - Groups that history into **cells**: buckets of past issues that share the same size and model, written `{ size, model }`. A forecast for an `L` issue on `claude-sonnet-4-6` is read off the `{ L, claude-sonnet-4-6 }` cell.
15
+ - Forecasts a **range**, not a single number: the **P50** (median — half of the cell's past issues cost at or below it) and the **P80** (80th percentile — 4 out of 5 did), for **tokens**, **turns**, **dollars**, and Codex **quota** (the fraction of your plan's rate-limit window the issue is predicted to use).
16
+ - Always reports **`n`** — how many past issues the forecast is based on — and flags a cell **low-confidence** when `n` is small. A forecast from 3 issues is barely a forecast.
17
+
18
+ It only *reads* telemetry and prints a forecast; it never modifies your usage records.
19
+
20
+ ## How good are the forecasts?
21
+
22
+ Be skeptical: a forecast is only as trustworthy as the history behind its `{ size, model }` cell, and in practice that history is thin — especially early on.
23
+
24
+ - **Most records carry no estimate.** Cost telemetry captures what an issue *spent*, but not its size; story-point estimates live in your tracker. Until they're joined onto the telemetry (`enrichUsageWithEstimate`) or stamped on when the work is dispatched, records have no `estimate` and can't be placed in any cell. A large telemetry file can still yield only a handful of usable issues.
25
+ - **Splitting by size *and* model fragments** what little estimate-tagged history you have across many small cells.
26
+
27
+ So expect small `n` and wide P50→P80 bands. **Treat the output as directional, not a budget** — useful for comparing relative cost between sizes or catching order-of-magnitude surprises, not for billing. Always read the printed `n` and `lowConfidence`; a single-digit `n` is a hint, not a number to plan against. The only thing that improves accuracy is more completed issues carrying estimates — no statistical trick manufactures signal the data doesn't have.
28
+
29
+ ## Install
30
+
31
+ ```bash
32
+ # One-shot via npx
33
+ npx llm-cost-estimate --size L --model claude-sonnet-4-6 --from-usage ./usage.jsonl
34
+
35
+ # Install globally
36
+ npm install -g llm-cost-estimation
37
+ llm-cost-estimate --size M --model gpt-5-codex --from-usage ./usage.jsonl
38
+ ```
39
+
40
+ ## CLI
41
+
42
+ ```bash
43
+ llm-cost-estimate --size <SIZE> --model <MODEL> [--from-usage <usage.jsonl-or-dir>] [--json]
44
+ llm-cost-estimate --issue <ID> --model <MODEL> [--from-usage <usage.jsonl-or-dir>] [--json]
45
+ llm-cost-estimate --help
46
+ ```
47
+
48
+ - `--size` takes the issue's size directly — a **story point** (the number, like 1/2/3/5/8, your tracker assigns to rate an issue's effort) or a **T-shirt size** (S/M/L/XL) — so it needs no tracker access.
49
+ - `--issue` resolves the estimate from your tracker through `createLinearEstimateSource` (requires `LINEAR_API_TOKEN`).
50
+ - `--from-usage` accepts a `usage.jsonl` file or a directory of `usage*.jsonl` files (same convention used by attribution backfill).
51
+ - `--json` prints machine-readable JSON.
52
+
53
+ ### Example
54
+
55
+ ```bash
56
+ llm-cost-estimate --size L --model claude-sonnet-4-6 --from-usage ./usage.jsonl
57
+ ```
58
+
59
+ ```text
60
+ ════════════════════════════════════════════════════════════════════════════════
61
+ COST FORECAST — size L, model claude-sonnet-4-6
62
+ ════════════════════════════════════════════════════════════════════════════════
63
+ Sample size: n = 18 (low confidence)
64
+
65
+ Metric P50 P80 n
66
+ ────────────────────────────────────────────────────────────────────────
67
+ tokens 1.2M 1.8M 18
68
+ turns 42 58 18
69
+ dollars $0.74 $1.01 18
70
+ quota (frac) 61.0% 68.5% 18
71
+ ```
72
+
73
+ **Dollars** here are *API-equivalent* — what those tokens would cost at pay-as-you-go API rates, not what a subscription plan is billed (the same convention `llm-cost-attribution` uses); on a subscription, the **quota** row is the one that reflects real marginal cost. `n = 18 (low confidence)` means only 18 past issues fell in this cell — read the range loosely.
74
+
75
+ JSON output:
76
+
77
+ ```bash
78
+ llm-cost-estimate --size 3 --model claude-sonnet-4-6 --from-usage ./usage.jsonl --json
79
+ ```
80
+
81
+ ```json
82
+ {
83
+ "size": "3",
84
+ "model": "claude-sonnet-4-6",
85
+ "n": 18,
86
+ "tokens": { "n": 18, "p50": 1215000, "p80": 1760000 },
87
+ "turns": { "n": 18, "p50": 42, "p80": 58 },
88
+ "dollars": { "n": 18, "p50": 0.74, "p80": 1.01 },
89
+ "quota": { "n": 18, "p50": 0.61, "p80": 0.685 },
90
+ "quotaReason": null,
91
+ "lowConfidence": true,
92
+ "empty": false
93
+ }
94
+ ```
95
+
96
+ ## Library API
97
+
98
+ ```js
99
+ import {
100
+ forecastIssueCost,
101
+ forecastProjectCost,
102
+ enrichUsageWithEstimate,
103
+ calibrate,
104
+ createLinearEstimateSource,
105
+ } from 'llm-cost-estimation';
106
+ ```
107
+
108
+ ### `forecastIssueCost(cell, records)`
109
+
110
+ Re-exported from [`llm-cost-attribution`](../llm-cost-attribution) for package consistency.
111
+
112
+ - `cell` is `{ size, model }`.
113
+ - `records` are estimate-tagged usage records (`{ estimate, model, ...tokens... }`).
114
+ - Returns a forecast object with P50/P80 + `n` for tokens, turns, dollars, and quota.
115
+
116
+ ### `enrichUsageWithEstimate(records, source, options?)`
117
+
118
+ Core transform for adding estimates to usage telemetry.
119
+
120
+ - Requires `source` implementing `resolveEstimates(issueIdentifiers): Map|string->number|null`.
121
+ - Adds `estimate` only when the source returns a valid non-negative integer.
122
+ - Returns `{ records, unresolved, stats }`.
123
+ - Issues with no estimate are left untouched and listed in `unresolved`.
124
+
125
+ ### `forecastProjectCost(projectId, issues, options?)`
126
+
127
+ Public API placeholder for project rollups.
128
+ Throws `Error('not implemented')` until the next sequencing issue lands.
129
+
130
+ ### `calibrate(completedIssues, options?)`
131
+
132
+ Public API placeholder for empirical calibration from completed work.
133
+ Throws `Error('not implemented')` until the next sequencing issue lands.
134
+
135
+ ## What it doesn't do
136
+
137
+ - It does **not** infer estimates from issue titles, paths, or code signals.
138
+ Add estimates in your tracker, then use `enrichUsageWithEstimate` to stamp them onto telemetry.
139
+ - It does **not** predict project-wide quota or wall-clock time.
140
+ - It does **not** promise accuracy from very thin cells.
141
+ A real forecast needs sufficient historical coverage in the exact `{ size, model }` cell;
142
+ low coverage is surfaced via `lowConfidence` and `n`.
143
+ - It does **not** merge multiple runs of the same issue for delivery quality.
144
+
145
+ The **quota** forecast is per-issue only — the peak fraction of Codex's primary rate-limit window a single issue is expected to hit. It does not add up across issues into a project-level quota.
146
+
147
+ ## License
148
+
149
+ MIT
@@ -0,0 +1,236 @@
1
+ #!/usr/bin/env node
2
+ /**
3
+ * `llm-cost-estimate` — forecast LLM cost for an issue before work begins.
4
+ *
5
+ * Two paths into a forecast:
6
+ * --size L --model X key-free; the cell is `{ size, model }`.
7
+ * --issue GRV-123 --model X reads the issue's story-point estimate from
8
+ * Linear (opt-in `LINEAR_API_TOKEN`), then
9
+ * forecasts at that size.
10
+ *
11
+ * Both paths read estimate-tagged usage records from `--from-usage <path>`
12
+ * (a single `usage.jsonl` file or a directory of `usage*.jsonl` files, per
13
+ * the Symphony Cost Telemetry Extension spec) and emit P50/P80 + n for
14
+ * tokens, turns, dollars, and — for Codex cells with rate_limits samples —
15
+ * the per-issue peak primary-window quota fraction. `--json` swaps the table
16
+ * for the same shape as JSON.
17
+ */
18
+ import { readUsageRecords } from 'llm-cost-attribution';
19
+ import { parseArgs } from 'node:util';
20
+ import { forecastIssueCost, createLinearEstimateSource } from '../src/index.mjs';
21
+
22
+ async function main() {
23
+ const { values } = parseArgs({
24
+ options: {
25
+ size: { type: 'string' },
26
+ issue: { type: 'string' },
27
+ model: { type: 'string' },
28
+ 'from-usage': { type: 'string' },
29
+ json: { type: 'boolean' },
30
+ help: { type: 'boolean', short: 'h' },
31
+ },
32
+ });
33
+
34
+ if (values.help === true) {
35
+ printUsage();
36
+ process.exit(0);
37
+ }
38
+ if (values.size === undefined && values.issue === undefined) {
39
+ printUsage();
40
+ process.exit(1);
41
+ }
42
+ if (values.size !== undefined && values.issue !== undefined) {
43
+ process.stderr.write('error: pass either --size or --issue, not both\n');
44
+ process.exit(1);
45
+ }
46
+ if (values.model === undefined || values.model === '') {
47
+ process.stderr.write('error: --model is required (e.g. --model claude-sonnet-4-6)\n');
48
+ process.exit(1);
49
+ }
50
+
51
+ const model = values.model;
52
+ let size;
53
+ let issueIdentifier;
54
+
55
+ if (values.size !== undefined) {
56
+ size = values.size;
57
+ } else {
58
+ issueIdentifier = values.issue;
59
+ const source = makeLinearEstimateSourceOrExit();
60
+ let resolved;
61
+ try {
62
+ resolved = await source.resolveEstimates([issueIdentifier]);
63
+ } catch (err) {
64
+ process.stderr.write(`error: failed to resolve estimate for ${issueIdentifier}: ${err.message}\n`);
65
+ process.exit(1);
66
+ }
67
+ const estimate = resolved instanceof Map
68
+ ? resolved.get(issueIdentifier)
69
+ : (resolved?.[issueIdentifier]);
70
+ if (estimate === null || estimate === undefined) {
71
+ process.stderr.write(`error: ${issueIdentifier} has no estimate in Linear; pass --size to forecast at a specific size\n`);
72
+ process.exit(1);
73
+ }
74
+ size = String(estimate);
75
+ }
76
+
77
+ const records = [];
78
+ if (values['from-usage'] !== undefined && values['from-usage'] !== '') {
79
+ for await (const record of readUsageRecords(values['from-usage'])) {
80
+ records.push(record);
81
+ }
82
+ }
83
+
84
+ const forecast = await forecastIssueCost({ size, model }, records);
85
+ const result = {
86
+ size,
87
+ model,
88
+ issueIdentifier,
89
+ n: forecast.tokens.n,
90
+ tokens: forecast.tokens,
91
+ turns: forecast.turns,
92
+ dollars: forecast.dollars,
93
+ quota: forecast.quota,
94
+ quotaReason: forecast.quotaReason,
95
+ lowConfidence: forecast.lowConfidence,
96
+ empty: forecast.empty,
97
+ };
98
+
99
+ if (values.json === true) {
100
+ console.log(JSON.stringify(result, null, 2));
101
+ return;
102
+ }
103
+
104
+ printTable(result);
105
+ }
106
+
107
+ function makeLinearEstimateSourceOrExit() {
108
+ try {
109
+ return createLinearEstimateSource();
110
+ } catch (err) {
111
+ process.stderr.write('error: set LINEAR_API_TOKEN or pass --size to forecast without a Linear lookup\n');
112
+ process.exit(1);
113
+ }
114
+ }
115
+
116
+ function printUsage() {
117
+ process.stdout.write(`Usage: llm-cost-estimate --size <SIZE> --model <MODEL> [--from-usage <path>] [--json]
118
+ llm-cost-estimate --issue <ID> --model <MODEL> [--from-usage <path>] [--json]
119
+ llm-cost-estimate --help
120
+
121
+ Forecast the expected LLM cost for an issue before work begins. The forecaster
122
+ matches \`{ size, model }\` cells against an estimate-tagged usage.jsonl dataset
123
+ and returns empirical P50/P80 quantiles for tokens, turns, dollars, and the
124
+ Codex primary-window quota fraction (single-issue only — never summed across
125
+ issues).
126
+
127
+ Inputs:
128
+ --size <SIZE> Story-point or T-shirt size to forecast at, e.g.
129
+ \`L\` or \`3\`. Key-free — no Linear lookup.
130
+ --issue <ID> Linear issue identifier, e.g. \`GRV-123\`. The CLI
131
+ resolves the issue's estimate via Linear (requires
132
+ \`LINEAR_API_TOKEN\`) and forecasts at that size.
133
+ --model <MODEL> Required. Model to forecast at, e.g.
134
+ \`claude-sonnet-4-6\` or \`gpt-5.4\`.
135
+ --from-usage <path> A \`usage.jsonl\` file or directory of \`usage*.jsonl\`
136
+ files (Symphony Cost Telemetry Extension spec). When
137
+ omitted the forecast is empty (n=0).
138
+ --json Emit JSON instead of the table.
139
+ -h, --help Print this message.
140
+
141
+ Examples:
142
+ llm-cost-estimate --size L --model claude-sonnet-4-6 --from-usage ~/usage.jsonl
143
+ llm-cost-estimate --issue GRV-123 --model claude-sonnet-4-6 --from-usage ~/usage.jsonl
144
+ llm-cost-estimate --size 3 --model gpt-5.4 --from-usage ~/usage.jsonl --json
145
+ `);
146
+ }
147
+
148
+ const HEAD = '═'.repeat(72);
149
+ const SEP = '─'.repeat(72);
150
+
151
+ function printTable(result) {
152
+ const cell = result.issueIdentifier !== undefined
153
+ ? `${result.issueIdentifier} (size ${result.size}, model ${result.model})`
154
+ : `size ${result.size}, model ${result.model}`;
155
+ console.log(HEAD);
156
+ console.log(`COST FORECAST — ${cell}`);
157
+ console.log(HEAD);
158
+ console.log(`Sample size: n = ${result.n}${result.lowConfidence ? ' (low confidence)' : ''}`);
159
+ if (result.empty) {
160
+ console.log();
161
+ console.log(`No historical issues match this cell — forecast is empty.`);
162
+ console.log(`Add more estimate-tagged records to --from-usage and try again.`);
163
+ return;
164
+ }
165
+ console.log();
166
+ console.log(
167
+ padRight('Metric', 14) +
168
+ ' ' + padLeft('P50', 12) +
169
+ ' ' + padLeft('P80', 12) +
170
+ ' ' + padLeft('n', 5),
171
+ );
172
+ console.log(SEP);
173
+ console.log(formatRow('tokens', result.tokens, formatTokens));
174
+ console.log(formatRow('turns', result.turns, formatTurns));
175
+ console.log(formatRow('dollars', result.dollars, formatUsd));
176
+ if (result.quota !== null && result.quota !== undefined) {
177
+ console.log(formatRow('quota (frac)', result.quota, formatFraction));
178
+ } else if (typeof result.quotaReason === 'string') {
179
+ console.log();
180
+ console.log(`(quota: ${result.quotaReason})`);
181
+ }
182
+ if (result.dollars !== null && result.dollars.n === 0 && result.n > 0) {
183
+ console.log();
184
+ console.log(`(no pricing rates for "${result.model}" — $ row reports n=0)`);
185
+ }
186
+ }
187
+
188
+ function formatRow(label, point, formatValue) {
189
+ if (point === null || point === undefined) {
190
+ return padRight(label, 14) + ' ' + padLeft('—', 12) + ' ' + padLeft('—', 12) + ' ' + padLeft('0', 5);
191
+ }
192
+ return (
193
+ padRight(label, 14) +
194
+ ' ' + padLeft(formatValue(point.p50), 12) +
195
+ ' ' + padLeft(formatValue(point.p80), 12) +
196
+ ' ' + padLeft(String(point.n), 5)
197
+ );
198
+ }
199
+
200
+ function formatTokens(value) {
201
+ if (value === null || value === undefined) return '—';
202
+ if (value >= 1_000_000) return `${(value / 1_000_000).toFixed(1)}M`;
203
+ if (value >= 1_000) return `${(value / 1_000).toFixed(1)}K`;
204
+ return String(Math.round(value));
205
+ }
206
+
207
+ function formatTurns(value) {
208
+ if (value === null || value === undefined) return '—';
209
+ return String(Math.round(value));
210
+ }
211
+
212
+ function formatUsd(value) {
213
+ if (value === null || value === undefined) return '—';
214
+ if (value === 0) return '$0.00';
215
+ if (value < 0.01) return '<$0.01';
216
+ if (value >= 1000) return `$${value.toLocaleString('en-US', { maximumFractionDigits: 0 })}`;
217
+ return `$${value.toFixed(2)}`;
218
+ }
219
+
220
+ function formatFraction(value) {
221
+ if (value === null || value === undefined) return '—';
222
+ return `${(value * 100).toFixed(1)}%`;
223
+ }
224
+
225
+ function padRight(s, width) {
226
+ return s.length >= width ? s : s + ' '.repeat(width - s.length);
227
+ }
228
+
229
+ function padLeft(s, width) {
230
+ return s.length >= width ? s : ' '.repeat(width - s.length) + s;
231
+ }
232
+
233
+ main().catch((err) => {
234
+ process.stderr.write(`${err.stack ?? err.message ?? String(err)}\n`);
235
+ process.exit(1);
236
+ });
package/package.json ADDED
@@ -0,0 +1,44 @@
1
+ {
2
+ "name": "llm-cost-estimation",
3
+ "version": "0.1.1",
4
+ "description": "Forecast LLM cost from Linear issue estimates before work begins.",
5
+ "type": "module",
6
+ "bin": {
7
+ "llm-cost-estimate": "bin/llm-cost-estimate.mjs"
8
+ },
9
+ "main": "./src/index.mjs",
10
+ "files": [
11
+ "bin/",
12
+ "src/",
13
+ "README.md",
14
+ "LICENSE"
15
+ ],
16
+ "scripts": {
17
+ "test": "node --test"
18
+ },
19
+ "keywords": [
20
+ "claude",
21
+ "claude-code",
22
+ "codex",
23
+ "anthropic",
24
+ "tokens",
25
+ "cost",
26
+ "estimation",
27
+ "forecast",
28
+ "agentic",
29
+ "autonomous-developer",
30
+ "symphony"
31
+ ],
32
+ "engines": {
33
+ "node": ">=20"
34
+ },
35
+ "dependencies": {
36
+ "llm-cost-attribution": "^0.2.0"
37
+ },
38
+ "license": "MIT",
39
+ "repository": {
40
+ "type": "git",
41
+ "url": "git+https://github.com/RiddimSoftware/groove.git",
42
+ "directory": "packages/llm-cost-estimation"
43
+ }
44
+ }
package/src/enrich.mjs ADDED
@@ -0,0 +1,132 @@
1
+ /**
2
+ * `EnrichUsageWithEstimate` use case.
3
+ *
4
+ * A pure transform: given cost-only usage records (as defined by the Symphony
5
+ * Coding-Agent Cost Telemetry Extension) and a `LinearEstimateSource` port,
6
+ * stamp each record with its issue's story-point `estimate` (spec §5.2).
7
+ *
8
+ * Boundary rule: this module MUST NOT import any Linear SDK or HTTP client. It
9
+ * depends only on the injected port so the estimation core stays key-free and
10
+ * tracker-agnostic — mirroring how `llm-cost-attribution` stays key-free.
11
+ *
12
+ * The port contract:
13
+ *
14
+ * source.resolveEstimates(issueIdentifiers: string[])
15
+ * => Map<string, number|null> | Record<string, number|null>
16
+ * (or a Promise of one)
17
+ *
18
+ * Resolve each distinct identifier to a non-negative integer estimate, or to
19
+ * `null` (or omit the key) when the issue has no estimate or no longer
20
+ * resolves. The core de-duplicates identifiers before calling, so the source
21
+ * sees at most one lookup per issue.
22
+ */
23
+
24
+ /**
25
+ * True for a spec-valid `estimate`: a non-negative integer (spec §5.2,
26
+ * "integer ≥ 0"). `0` is a real estimate value, so it passes — only `null`,
27
+ * `undefined`, fractional, or negative values are rejected.
28
+ *
29
+ * @param {unknown} value
30
+ * @returns {boolean}
31
+ */
32
+ export function isValidEstimate(value) {
33
+ return typeof value === 'number' && Number.isInteger(value) && value >= 0;
34
+ }
35
+
36
+ /**
37
+ * Look up an estimate for an identifier from whatever the source returned. The
38
+ * source MAY return a `Map` or a plain object; absent keys and non-own
39
+ * properties are treated as unresolved (`null`).
40
+ *
41
+ * @param {Map<string, unknown> | Record<string, unknown> | null | undefined} resolved
42
+ * @param {string} id
43
+ * @returns {unknown}
44
+ */
45
+ function lookupEstimate(resolved, id) {
46
+ if (resolved == null) return null;
47
+ if (resolved instanceof Map) return resolved.has(id) ? resolved.get(id) : null;
48
+ if (Object.prototype.hasOwnProperty.call(resolved, id)) return resolved[id];
49
+ return null;
50
+ }
51
+
52
+ /**
53
+ * Stamp the spec's optional `estimate` field onto each usage record.
54
+ *
55
+ * Distinct `issueIdentifier`s are de-duplicated before the source is queried,
56
+ * so the port sees at most one lookup per issue. Records whose issue resolves
57
+ * to a non-negative integer estimate gain `estimate`; all other fields are left
58
+ * unchanged. Records whose issue has no estimate (`null`) or no longer resolves
59
+ * are returned untouched — `estimate` stays **absent, never `0`** — and the
60
+ * issue identifier is reported in the `unresolved` summary.
61
+ *
62
+ * Input records are never mutated; a shallow copy is returned for each.
63
+ *
64
+ * @param {Iterable<object>} records Usage records (typically estimate-free).
65
+ * @param {{ resolveEstimates: (ids: string[]) => unknown }} source A `LinearEstimateSource`.
66
+ * @param {object} [options] Reserved for future options.
67
+ * @returns {Promise<{
68
+ * records: object[],
69
+ * unresolved: string[],
70
+ * stats: {
71
+ * recordsTotal: number,
72
+ * recordsEnriched: number,
73
+ * issuesQueried: number,
74
+ * issuesResolved: number,
75
+ * issuesUnresolved: number,
76
+ * },
77
+ * }>}
78
+ */
79
+ export async function enrichUsageWithEstimate(records, source, options = {}) {
80
+ if (source == null || typeof source.resolveEstimates !== 'function') {
81
+ throw new TypeError(
82
+ 'enrichUsageWithEstimate: source must implement resolveEstimates(ids)',
83
+ );
84
+ }
85
+
86
+ const input = [...records];
87
+
88
+ // De-duplicate distinct issue identifiers so the source is queried at most
89
+ // once per issue (≤1 lookup per issue, batched where the API allows).
90
+ const distinctIds = [
91
+ ...new Set(
92
+ input
93
+ .map((rec) => rec?.issueIdentifier)
94
+ .filter((id) => typeof id === 'string' && id !== ''),
95
+ ),
96
+ ];
97
+
98
+ const resolved = distinctIds.length > 0
99
+ ? await source.resolveEstimates(distinctIds)
100
+ : new Map();
101
+
102
+ const resolvedIds = new Set();
103
+ const unresolvedIds = new Set();
104
+ let recordsEnriched = 0;
105
+
106
+ const out = input.map((rec) => {
107
+ const id = rec?.issueIdentifier;
108
+ if (typeof id !== 'string' || id === '') {
109
+ return { ...rec };
110
+ }
111
+ const estimate = lookupEstimate(resolved, id);
112
+ if (isValidEstimate(estimate)) {
113
+ resolvedIds.add(id);
114
+ recordsEnriched += 1;
115
+ return { ...rec, estimate };
116
+ }
117
+ unresolvedIds.add(id);
118
+ return { ...rec };
119
+ });
120
+
121
+ return {
122
+ records: out,
123
+ unresolved: [...unresolvedIds].sort(),
124
+ stats: {
125
+ recordsTotal: out.length,
126
+ recordsEnriched,
127
+ issuesQueried: distinctIds.length,
128
+ issuesResolved: resolvedIds.size,
129
+ issuesUnresolved: unresolvedIds.size,
130
+ },
131
+ };
132
+ }
package/src/index.mjs ADDED
@@ -0,0 +1,55 @@
1
+ /**
2
+ * Public API for `llm-cost-estimation`.
3
+ *
4
+ * Implemented exports are re-exported from their sub-modules; the remaining
5
+ * stubs throw until their implementing issue lands. Import from this barrel —
6
+ * do not import from sub-modules directly.
7
+ */
8
+
9
+ /**
10
+ * Stamp the Symphony Cost Telemetry Extension's optional `estimate` field onto
11
+ * usage records by joining each record's issue to its Linear story-point
12
+ * estimate via an injected `LinearEstimateSource` port. Pure transform — see
13
+ * `enrich.mjs`.
14
+ */
15
+ export { enrichUsageWithEstimate, isValidEstimate } from './enrich.mjs';
16
+
17
+ /**
18
+ * Linear-backed `LinearEstimateSource` adapter for `enrichUsageWithEstimate`.
19
+ * Reads the API token from an injected option or `LINEAR_API_TOKEN`.
20
+ */
21
+ export { createLinearEstimateSource } from './linear-estimate-source.mjs';
22
+
23
+ /**
24
+ * Forecast tokens / turns / dollars / quota P50–P80 for a `{ size, model }`
25
+ * cell from a set of estimate-tagged usage records. Re-exported from
26
+ * `llm-cost-attribution`, which owns the empirical-quantile forecaster and
27
+ * its `PricingTable` / `QuotaModel` adapters.
28
+ */
29
+ export { forecastIssueCost } from 'llm-cost-attribution';
30
+
31
+ /**
32
+ * Forecast the aggregate LLM cost for an entire Linear project, given per-issue
33
+ * estimates and a calibration dataset.
34
+ *
35
+ * @param {string} projectId Linear project identifier.
36
+ * @param {object[]} issues Array of `{ identifier, estimate }` objects.
37
+ * @param {object} [options]
38
+ * @returns {Promise<object>}
39
+ */
40
+ export async function forecastProjectCost(projectId, issues, options = {}) {
41
+ throw new Error('not implemented');
42
+ }
43
+
44
+ /**
45
+ * Build or update a calibration dataset from a set of completed issues whose
46
+ * actual cost is known. Returns calibration parameters used by the forecast
47
+ * functions.
48
+ *
49
+ * @param {object[]} completedIssues Array of `{ identifier, estimate, actualCostUsd }`.
50
+ * @param {object} [options]
51
+ * @returns {object}
52
+ */
53
+ export function calibrate(completedIssues, options = {}) {
54
+ throw new Error('not implemented');
55
+ }
@@ -0,0 +1,131 @@
1
+ /**
2
+ * Linear adapter implementing the `LinearEstimateSource` port consumed by the
3
+ * `EnrichUsageWithEstimate` use case (`enrich.mjs`).
4
+ *
5
+ * This is the ONLY module in the package that talks to Linear. The enrichment
6
+ * core depends on the port, not on this adapter, so the core stays key-free and
7
+ * tracker-agnostic. The API token is read from an injected option or the
8
+ * `LINEAR_API_TOKEN` environment variable — it is never hardcoded, logged, or
9
+ * written to any usage record (spec §8).
10
+ *
11
+ * Tests inject a fake source instead of this adapter; there are no live Linear
12
+ * calls in CI.
13
+ */
14
+
15
+ const LINEAR_GRAPHQL_ENDPOINT = 'https://api.linear.app/graphql';
16
+
17
+ // Linear caps `first` at 250; one (teamKey, number) filter matches at most one
18
+ // issue, so a chunk of this size returns in a single page with no pagination.
19
+ const DEFAULT_CHUNK_SIZE = 100;
20
+
21
+ const IDENTIFIER_PATTERN = /^([A-Za-z][A-Za-z0-9]*)-(\d+)$/;
22
+
23
+ const ESTIMATES_QUERY = `query IssueEstimates($filter: IssueFilter, $first: Int) {
24
+ issues(filter: $filter, first: $first) {
25
+ nodes { identifier estimate }
26
+ }
27
+ }`;
28
+
29
+ /**
30
+ * Split `"EPAC-1999"` into `{ teamKey: "EPAC", number: 1999 }`, or `null` if it
31
+ * isn't a `<TEAM>-<NUMBER>` identifier.
32
+ *
33
+ * @param {string} identifier
34
+ */
35
+ function parseIdentifier(identifier) {
36
+ const match = IDENTIFIER_PATTERN.exec(identifier);
37
+ if (match === null) return null;
38
+ return { teamKey: match[1], number: Number(match[2]) };
39
+ }
40
+
41
+ function chunk(items, size) {
42
+ const out = [];
43
+ for (let i = 0; i < items.length; i += size) {
44
+ out.push(items.slice(i, i + size));
45
+ }
46
+ return out;
47
+ }
48
+
49
+ /**
50
+ * Create a `LinearEstimateSource` backed by Linear's GraphQL API.
51
+ *
52
+ * @param {object} [options]
53
+ * @param {string} [options.token] Linear API token. Defaults to `process.env.LINEAR_API_TOKEN`.
54
+ * @param {string} [options.endpoint] GraphQL endpoint. Defaults to Linear's production endpoint.
55
+ * @param {typeof fetch} [options.fetch] Fetch implementation. Defaults to the global `fetch`.
56
+ * @param {number} [options.chunkSize] Max identifiers per GraphQL request.
57
+ * @returns {{ resolveEstimates: (issueIdentifiers: string[]) => Promise<Map<string, number|null>> }}
58
+ */
59
+ export function createLinearEstimateSource(options = {}) {
60
+ const token = options.token ?? process.env.LINEAR_API_TOKEN;
61
+ if (typeof token !== 'string' || token === '') {
62
+ throw new Error(
63
+ 'createLinearEstimateSource: a Linear API token is required (pass options.token or set LINEAR_API_TOKEN)',
64
+ );
65
+ }
66
+ const endpoint = options.endpoint ?? LINEAR_GRAPHQL_ENDPOINT;
67
+ const fetchImpl = options.fetch ?? globalThis.fetch;
68
+ if (typeof fetchImpl !== 'function') {
69
+ throw new TypeError('createLinearEstimateSource: no fetch implementation available');
70
+ }
71
+ const chunkSize = options.chunkSize ?? DEFAULT_CHUNK_SIZE;
72
+
73
+ async function fetchChunk(parsed) {
74
+ const filter = {
75
+ or: parsed.map(({ teamKey, number }) => ({
76
+ team: { key: { eq: teamKey } },
77
+ number: { eq: number },
78
+ })),
79
+ };
80
+ const res = await fetchImpl(endpoint, {
81
+ method: 'POST',
82
+ headers: {
83
+ 'content-type': 'application/json',
84
+ authorization: token,
85
+ },
86
+ body: JSON.stringify({
87
+ query: ESTIMATES_QUERY,
88
+ variables: { filter, first: parsed.length },
89
+ }),
90
+ });
91
+ if (!res.ok) {
92
+ throw new Error(`Linear API request failed: HTTP ${res.status}`);
93
+ }
94
+ const json = await res.json();
95
+ if (json.errors) {
96
+ throw new Error(`Linear API returned errors: ${JSON.stringify(json.errors)}`);
97
+ }
98
+ return json?.data?.issues?.nodes ?? [];
99
+ }
100
+
101
+ return {
102
+ /**
103
+ * Resolve each distinct identifier to a non-negative integer estimate, or
104
+ * `null` when the issue has no estimate or no longer resolves. Identifiers
105
+ * are de-duplicated by the caller; this method assumes they are distinct.
106
+ */
107
+ async resolveEstimates(issueIdentifiers) {
108
+ const result = new Map();
109
+ const parsed = [];
110
+ for (const id of issueIdentifiers) {
111
+ const p = parseIdentifier(id);
112
+ if (p === null) {
113
+ result.set(id, null); // unparseable → unresolved
114
+ } else {
115
+ parsed.push({ id, ...p });
116
+ }
117
+ }
118
+
119
+ for (const group of chunk(parsed, chunkSize)) {
120
+ const nodes = await fetchChunk(group);
121
+ const byIdentifier = new Map(nodes.map((n) => [n.identifier, n.estimate]));
122
+ for (const { id } of group) {
123
+ const estimate = byIdentifier.has(id) ? byIdentifier.get(id) : null;
124
+ result.set(id, estimate ?? null);
125
+ }
126
+ }
127
+
128
+ return result;
129
+ },
130
+ };
131
+ }
@@ -0,0 +1,91 @@
1
+ /**
2
+ * Leak-safety guard for the llm-cost-estimation package.
3
+ *
4
+ * Fixtures here derive from real telemetry, so a slipped `/Users/<name>` path
5
+ * or private repo name would publish org/personal data on npm. This module
6
+ * exposes the deny patterns and a string scanner; `test/no-org-data.test.mjs`
7
+ * wires them into `npm test` so a leak fails CI rather than relying on
8
+ * reviewer vigilance.
9
+ *
10
+ * Allowed: opaque tracker IDs matching `[A-Z]+-\d+` on their own (e.g.
11
+ * `EPAC-1999`, `GRV-42`).
12
+ * Denied: absolute home paths (`/Users/<name>`, `/home/<name>`), the
13
+ * `.symphony/workspaces/<ID>` path shape, and any listed private repo name.
14
+ */
15
+
16
+ // Configurable list of private repo names. Extend this array when a new repo
17
+ // joins the org's "private" set; the guard will start flagging the name as a
18
+ // whole-word match.
19
+ export const PRIVATE_REPO_NAMES = Object.freeze([]);
20
+
21
+ // Built-in deny rules. Each regex MUST use the `g` flag — `scanText` walks
22
+ // every match on a line.
23
+ export const DEFAULT_DENY_PATTERNS = Object.freeze([
24
+ {
25
+ name: 'home-path-users',
26
+ regex: /\/Users\/[A-Za-z0-9._-]+/g,
27
+ description: 'absolute /Users/<name> home path (macOS personal data)',
28
+ },
29
+ {
30
+ name: 'home-path-home',
31
+ regex: /\/home\/[A-Za-z0-9._-]+/g,
32
+ description: 'absolute /home/<name> path (Linux personal data)',
33
+ },
34
+ {
35
+ name: 'symphony-workspace-path',
36
+ regex: /\.symphony\/workspaces\/[A-Za-z0-9._-]+/g,
37
+ description: '.symphony/workspaces/<ID> path (private orchestrator state)',
38
+ },
39
+ ]);
40
+
41
+ /**
42
+ * Build a deny-pattern set, optionally extending the built-ins with a list of
43
+ * private repo names. Each repo name is matched as a whole word so a name
44
+ * like `epac` does not match `epacenter`.
45
+ */
46
+ export function buildDenyPatterns({ privateRepoNames = PRIVATE_REPO_NAMES } = {}) {
47
+ const patterns = [...DEFAULT_DENY_PATTERNS];
48
+ for (const name of privateRepoNames) {
49
+ patterns.push({
50
+ name: `private-repo:${name}`,
51
+ regex: new RegExp(`\\b${escapeRegex(name)}\\b`, 'g'),
52
+ description: `private repo name "${name}"`,
53
+ });
54
+ }
55
+ return patterns;
56
+ }
57
+
58
+ function escapeRegex(s) {
59
+ return s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
60
+ }
61
+
62
+ /**
63
+ * Scan a string for leaked data. Returns an array of findings; an empty
64
+ * array means clean. Findings carry 1-indexed line + column so callers can
65
+ * print human-readable file:line:col error messages.
66
+ *
67
+ * @returns {{ line: number, column: number, match: string, patternName: string, description: string }[]}
68
+ */
69
+ export function scanText(text, patterns = buildDenyPatterns()) {
70
+ const findings = [];
71
+ const lines = text.split('\n');
72
+ for (let i = 0; i < lines.length; i++) {
73
+ const line = lines[i];
74
+ for (const pattern of patterns) {
75
+ // Shared regex objects carry lastIndex state across calls; reset it.
76
+ pattern.regex.lastIndex = 0;
77
+ let m;
78
+ while ((m = pattern.regex.exec(line)) !== null) {
79
+ findings.push({
80
+ line: i + 1,
81
+ column: m.index + 1,
82
+ match: m[0],
83
+ patternName: pattern.name,
84
+ description: pattern.description,
85
+ });
86
+ if (m.index === pattern.regex.lastIndex) pattern.regex.lastIndex++;
87
+ }
88
+ }
89
+ }
90
+ return findings;
91
+ }