sigmap 8.21.0 → 8.23.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +25 -0
- package/README.md +11 -11
- package/gen-context.js +317 -6
- package/llms-full.txt +19 -7
- package/llms.txt +6 -6
- package/package.json +1 -1
- package/packages/cli/package.json +1 -1
- package/packages/core/package.json +1 -1
- package/src/config/defaults.js +8 -0
- package/src/eval/corpus.js +83 -0
- package/src/eval/runner.js +2 -1
- package/src/mcp/handlers.js +36 -1
- package/src/mcp/server.js +3 -2
- package/src/mcp/tools.js +26 -0
- package/src/tracking/budget.js +113 -0
- package/src/tracking/logger.js +3 -1
- package/src/tracking/pricing.js +2 -0
package/CHANGELOG.md
CHANGED
|
@@ -10,6 +10,31 @@ Format: [Semantic Versioning](https://semver.org/)
|
|
|
10
10
|
|
|
11
11
|
---
|
|
12
12
|
|
|
13
|
+
## [8.23.0] — 2026-07-28
|
|
14
|
+
|
|
15
|
+
Minor release — **"Agent Economy I" (v8.23, F1)**: SigMap's token savings become queryable *during* a session. A spend ledger over the existing gain log, an optional budget threshold, and context-freshness age — as a CLI command and the 21st MCP tool.
|
|
16
|
+
|
|
17
|
+
### Added
|
|
18
|
+
- **`sigmap budget` + MCP `get_budget` (#508, PR #509):** new `src/tracking/budget.js` — `budgetStatus()` sums the session's gain-log entries (estimated SigMap-emitted tokens: spent/baseline/saved, op count), computes remaining/pct against an optional budget, and reports generated-context age with a stale flag. Session identity: `SIGMAP_SESSION` env override, else UTC day bucket; `recordUsage` now stamps every entry with `session` (legacy entries match day buckets by timestamp prefix). New opt-in config keys `sessionBudgetTokens` and `contextTtlDays` (both default `null`). CLI: `sigmap budget [--json] [--session <key>] [--budget <tokens>]`. The MCP tool advises degrade-gracefully tactics (terse encoding, `squeeze`, summarize-then-drop) at ≥80% budget. 7 new integration tests (129 test files); tool-count guards advanced 20→21.
|
|
19
|
+
|
|
20
|
+
### Changed
|
|
21
|
+
- **Scope honesty by design:** the ledger counts tokens **SigMap emitted** (chars/4, labeled `estimated-tokens` on every surface) — not the host chat's total spend, which a CLI cannot see. Context freshness replaces the originally-planned prompt-cache "injection TTL", which was unverifiable and got cut in the plan audit.
|
|
22
|
+
|
|
23
|
+
---
|
|
24
|
+
|
|
25
|
+
## [8.22.0] — 2026-07-28
|
|
26
|
+
|
|
27
|
+
Minor release — **"Hard Corpus" (v8.22)**: the benchmark corpus gains a no-leakage hard split with a deterministic leakage gate, and per-repo-size buckets stop tiny repos from flattering the average. The headline retrieval number gets harder — and honest.
|
|
28
|
+
|
|
29
|
+
### Added
|
|
30
|
+
- **Hard-split corpus + leakage gate + size buckets (#505, PR #506):** new `src/eval/corpus.js` — a task "leaks" when its BM25-tokenized query shares a stemmed token with the tokenized basenames of its expected files; `validateTasks` flags leaky `split: "hard"` tasks as violations, and `sizeBucket` groups repos at 200/1000 scanned files (tertiles of the 43-repo corpus). New CI gate `scripts/validate-task-corpus.mjs` (exit 1 on hard-split leakage). `loadTasks` carries the optional `split` field (default `easy`). `benchmark:honest` now reports hit@5/MRR per split and per size bucket — buckets use files scanned on disk, not the budget-capped context index. 15 hand-authored hard tasks (express, flask, axios, fastify, gin), all leak-free. 8 new integration tests (128 test files).
|
|
31
|
+
- **MiniMax LLM-ablation provider (PR #504)** — thanks @octo-patch: `MINIMAX_API_KEY` support in `scripts/run-llm-ablation.mjs` (OpenAI-compatible endpoint, default model MiniMax-M3, `MINIMAX_BASE_URL` override) plus a pricing entry and tests.
|
|
32
|
+
|
|
33
|
+
### Changed
|
|
34
|
+
- **Headline honesty, again:** the leakage gate measured that **90 of 110 pre-existing easy tasks leak filename tokens**, and the new hard split scores **33.3% hit@5 vs the grep baseline's 53.3%** — with leakage removed, grep currently wins; that measured vocabulary-mismatch ceiling is what B2 (repo-mined expansion, v9.0) exists to attack. Overall corpus (125 tasks): 72.8% hit@5, honest lift 1.63× (+28pt).
|
|
35
|
+
|
|
36
|
+
---
|
|
37
|
+
|
|
13
38
|
## [8.21.0] — 2026-07-19
|
|
14
39
|
|
|
15
40
|
Minor release — **"Semantic Bridge II" (v8.21)**: doc-comment hints reach Go, Rust, and Java, and the import graph gains a principled centrality prior for ranking — flag-gated and measured.
|
package/README.md
CHANGED
|
@@ -59,10 +59,10 @@ That map is exactly what agentic grep is worst at: reproducible, auditable conte
|
|
|
59
59
|
|
|
60
60
|
**Proof it pays off** (full benchmark below):
|
|
61
61
|
<!--SM:whyMetrics-->
|
|
62
|
-
- **
|
|
62
|
+
- **82.2% hit@5** — right file in top 5 results (vs 44.8% single-shot grep baseline — 1.59× lift)
|
|
63
63
|
- **96.8% token reduction** — average across 21 real repos
|
|
64
|
-
- **
|
|
65
|
-
- **1.
|
|
64
|
+
- **64.8% task-success proxy** — modeled from retrieval tiers, not measured LLM sessions
|
|
65
|
+
- **1.53 prompts per task** — down from 2.84 (46.1% fewer retries, modeled)
|
|
66
66
|
<!--/SM:whyMetrics-->
|
|
67
67
|
- **<!--SM:languages-->33<!--/SM:languages--> languages supported** — TypeScript, Python, Go, Rust, Java, R, and more
|
|
68
68
|
- **No vendor lock-in** — works with any AI assistant or local LLM
|
|
@@ -98,7 +98,7 @@ sigmap verify answer.md --report # standalone red/amber/green HTML report
|
|
|
98
98
|
| Without SigMap | With SigMap |
|
|
99
99
|
|---|---|
|
|
100
100
|
| ❌ Non-reproducible agent guesses | ✅ Deterministic map — same input, same output, every time |
|
|
101
|
-
| ❌ "Trust me" AI answers | ✅ Grounded — right file in context <!--SM:hitWhole-->
|
|
101
|
+
| ❌ "Trust me" AI answers | ✅ Grounded — right file in context <!--SM:hitWhole-->82%<!--/SM:hitWhole--> of the time, every symbol on a real line anchor |
|
|
102
102
|
| ❌ Embeddings / vector DB required | ✅ Zero deps, no infra, fully offline |
|
|
103
103
|
|
|
104
104
|
---
|
|
@@ -122,13 +122,13 @@ Ask → Rank → Context → Validate → Judge → Learn
|
|
|
122
122
|
|
|
123
123
|
<!--SM:benchmarkBlock-->
|
|
124
124
|
```
|
|
125
|
-
Benchmark : sigmap-v8.
|
|
126
|
-
Date : 2026-07-
|
|
125
|
+
Benchmark : sigmap-v8.23-main (21 repositories, including R language)
|
|
126
|
+
Date : 2026-07-28
|
|
127
127
|
|
|
128
|
-
Hit@5 :
|
|
128
|
+
Hit@5 : 82.2% (grep-agent baseline 44.8% — 1.59× lift)
|
|
129
129
|
Token reduction: 96.8% (across 21 repos)
|
|
130
|
-
Prompt reduction :
|
|
131
|
-
Task success :
|
|
130
|
+
Prompt reduction : 46.1% (2.84 → 1.53 prompts per task, modeled)
|
|
131
|
+
Task success : 64.8% (proxy — modeled from retrieval tiers)
|
|
132
132
|
Repos tested : 21 (JavaScript, Python, Go, Rust, Java, R, C++, C#, Dart, Swift, Ruby, PHP, Scala, Kotlin, and more)
|
|
133
133
|
```
|
|
134
134
|
<!--/SM:benchmarkBlock-->
|
|
@@ -222,7 +222,7 @@ Use SigMap with open-source tools and fully self-hosted setups:
|
|
|
222
222
|
| **JetBrains** | [Marketplace](https://plugins.jetbrains.com/plugin/31109-sigmap--ai-context-engine/) | [github.com/manojmallick/sigmap-jetbrains](https://github.com/manojmallick/sigmap-jetbrains) | IntelliJ IDEA, WebStorm, PyCharm, GoLand — tool window + actions |
|
|
223
223
|
| **Neovim** | lazy.nvim / packer / vim-plug | [github.com/manojmallick/sigmap.nvim](https://github.com/manojmallick/sigmap.nvim) | `:SigMap`, `:SigMapQuery` float window, statusline widget |
|
|
224
224
|
|
|
225
|
-
**MCP server** —
|
|
225
|
+
**MCP server** — 21 on-demand tools for Claude Code and Cursor:
|
|
226
226
|
|
|
227
227
|
```bash
|
|
228
228
|
sigmap --mcp
|
|
@@ -269,7 +269,7 @@ SigMap treats coding agents as **consumers, not competitors**: it hands them a d
|
|
|
269
269
|
|
|
270
270
|
| Agent | One-time setup | How it consumes SigMap |
|
|
271
271
|
|---|---|---|
|
|
272
|
-
| **Claude Code** | `sigmap mcp install claude` |
|
|
272
|
+
| **Claude Code** | `sigmap mcp install claude` | 21 MCP tools (`search_signatures`, `get_lines`, `get_diff_context`, `squeeze_output`…) |
|
|
273
273
|
| **Cursor** | `sigmap mcp install cursor` | MCP tools, plus the `cursor` adapter writes `.cursorrules` |
|
|
274
274
|
| **Cline** | `sigmap mcp install cursor` | Reads `.cursorrules`; same MCP server |
|
|
275
275
|
| **Continue** | `sigmap mcp install vscode` | MCP tools inside the Continue extension |
|
package/gen-context.js
CHANGED
|
@@ -1490,6 +1490,14 @@ __factories["./src/config/defaults"] = function(module, exports) {
|
|
|
1490
1490
|
// Append run metrics to .context/usage.ndjson after each generate
|
|
1491
1491
|
tracking: false,
|
|
1492
1492
|
|
|
1493
|
+
// Session spend ledger (`sigmap budget` / MCP get_budget). Estimates only —
|
|
1494
|
+
// counts tokens SigMap emitted (chars/4), not the host chat's total spend.
|
|
1495
|
+
// Number → warn threshold for estimated SigMap-emitted tokens per session.
|
|
1496
|
+
sessionBudgetTokens: null,
|
|
1497
|
+
|
|
1498
|
+
// Number of days before generated context counts as stale in budget output.
|
|
1499
|
+
contextTtlDays: null,
|
|
1500
|
+
|
|
1493
1501
|
// MCP server configuration
|
|
1494
1502
|
mcp: {
|
|
1495
1503
|
autoRegister: true,
|
|
@@ -4101,6 +4109,93 @@ __factories["./src/eval/analyzer"] = function(module, exports) {
|
|
|
4101
4109
|
|
|
4102
4110
|
};
|
|
4103
4111
|
|
|
4112
|
+
// ── ./src/eval/corpus ──
|
|
4113
|
+
__factories["./src/eval/corpus"] = function(module, exports) {
|
|
4114
|
+
|
|
4115
|
+
/**
|
|
4116
|
+
* Task-corpus hygiene (A3, v8.22 "Hard Corpus").
|
|
4117
|
+
*
|
|
4118
|
+
* A benchmark query "leaks" when it shares a token with the basenames of its
|
|
4119
|
+
* expected files — hit@5 then partly measures filename matching, not
|
|
4120
|
+
* retrieval. The criterion is deterministic and reuses the production
|
|
4121
|
+
* tokenizer (identifier splitting + stemming from src/retrieval/bm25.js), so
|
|
4122
|
+
* "payments" leaks against payment.js and "InterceptorManager" leaks against
|
|
4123
|
+
* "interceptor manager" the same way the ranker would see them.
|
|
4124
|
+
*
|
|
4125
|
+
* Tasks carry an optional `split` field: 'hard' tasks MUST be leak-free
|
|
4126
|
+
* (validateTasks reports them as violations); 'easy' tasks (the default) may
|
|
4127
|
+
* leak — that is what makes them easy.
|
|
4128
|
+
*
|
|
4129
|
+
* Size buckets group repos by indexed file count so large repos stop being
|
|
4130
|
+
* averaged away by tiny ones. Thresholds are the rough tertiles of the
|
|
4131
|
+
* current benchmarks/repos corpus (43 repos, 27–3450 source files).
|
|
4132
|
+
*/
|
|
4133
|
+
|
|
4134
|
+
const { tokenize } = __require('./src/retrieval/bm25');
|
|
4135
|
+
|
|
4136
|
+
const BUCKET_LIMITS = { small: 200, medium: 1000 }; // files; large = above medium
|
|
4137
|
+
|
|
4138
|
+
/**
|
|
4139
|
+
* Stemmed tokens of a file path's basename (extension stripped).
|
|
4140
|
+
* @param {string} filePath
|
|
4141
|
+
* @returns {string[]}
|
|
4142
|
+
*/
|
|
4143
|
+
function basenameTokens(filePath) {
|
|
4144
|
+
const base = String(filePath).split('/').pop() || '';
|
|
4145
|
+
return tokenize(base.replace(/\.[^.]*$/, ''));
|
|
4146
|
+
}
|
|
4147
|
+
|
|
4148
|
+
/**
|
|
4149
|
+
* Leaked tokens between a query and its expected files' basenames.
|
|
4150
|
+
* @param {string} query
|
|
4151
|
+
* @param {string[]} expectedFiles
|
|
4152
|
+
* @returns {{ leaked: string[], clean: boolean }}
|
|
4153
|
+
*/
|
|
4154
|
+
function queryLeakage(query, expectedFiles) {
|
|
4155
|
+
const qToks = new Set(tokenize(query));
|
|
4156
|
+
const leaked = new Set();
|
|
4157
|
+
for (const f of expectedFiles || []) {
|
|
4158
|
+
for (const t of basenameTokens(f)) {
|
|
4159
|
+
if (qToks.has(t)) leaked.add(t);
|
|
4160
|
+
}
|
|
4161
|
+
}
|
|
4162
|
+
return { leaked: [...leaked].sort(), clean: leaked.size === 0 };
|
|
4163
|
+
}
|
|
4164
|
+
|
|
4165
|
+
/**
|
|
4166
|
+
* Validate a task list: every task gets a leakage result; hard-split tasks
|
|
4167
|
+
* that leak are violations.
|
|
4168
|
+
* @param {Array<{id?:string, query:string, expected_files?:string[], split?:string}>} tasks
|
|
4169
|
+
* @returns {{ results: object[], hardViolations: object[] }}
|
|
4170
|
+
*/
|
|
4171
|
+
function validateTasks(tasks) {
|
|
4172
|
+
const results = [];
|
|
4173
|
+
const hardViolations = [];
|
|
4174
|
+
for (const t of tasks || []) {
|
|
4175
|
+
const split = t.split === 'hard' ? 'hard' : 'easy';
|
|
4176
|
+
const { leaked, clean } = queryLeakage(t.query, t.expected_files);
|
|
4177
|
+
const row = { id: t.id || '?', split, leaked, clean };
|
|
4178
|
+
results.push(row);
|
|
4179
|
+
if (split === 'hard' && !clean) hardViolations.push(row);
|
|
4180
|
+
}
|
|
4181
|
+
return { results, hardViolations };
|
|
4182
|
+
}
|
|
4183
|
+
|
|
4184
|
+
/**
|
|
4185
|
+
* Size bucket for a repo by indexed file count.
|
|
4186
|
+
* @param {number} fileCount
|
|
4187
|
+
* @returns {'small'|'medium'|'large'}
|
|
4188
|
+
*/
|
|
4189
|
+
function sizeBucket(fileCount) {
|
|
4190
|
+
if (fileCount < BUCKET_LIMITS.small) return 'small';
|
|
4191
|
+
if (fileCount <= BUCKET_LIMITS.medium) return 'medium';
|
|
4192
|
+
return 'large';
|
|
4193
|
+
}
|
|
4194
|
+
|
|
4195
|
+
module.exports = { basenameTokens, queryLeakage, validateTasks, sizeBucket, BUCKET_LIMITS };
|
|
4196
|
+
|
|
4197
|
+
};
|
|
4198
|
+
|
|
4104
4199
|
// ── ./src/eval/llm-ablation ──
|
|
4105
4200
|
__factories["./src/eval/llm-ablation"] = function(module, exports) {
|
|
4106
4201
|
|
|
@@ -4400,7 +4495,7 @@ __factories["./src/eval/runner"] = function(module, exports) {
|
|
|
4400
4495
|
|
|
4401
4496
|
/**
|
|
4402
4497
|
* Load tasks from a JSONL file.
|
|
4403
|
-
* Each line: { id, query, expected_files, repo }
|
|
4498
|
+
* Each line: { id, query, expected_files, repo, split? ('easy'|'hard') }
|
|
4404
4499
|
* Invalid or blank lines are silently skipped.
|
|
4405
4500
|
* @param {string} tasksFile - absolute or relative path
|
|
4406
4501
|
* @returns {Array<{id:string, query:string, expected:string[], repo:string}>}
|
|
@@ -4420,6 +4515,7 @@ __factories["./src/eval/runner"] = function(module, exports) {
|
|
|
4420
4515
|
query: obj.query,
|
|
4421
4516
|
expected: obj.expected_files,
|
|
4422
4517
|
repo: obj.repo || '.',
|
|
4518
|
+
split: obj.split === 'hard' ? 'hard' : 'easy',
|
|
4423
4519
|
});
|
|
4424
4520
|
}
|
|
4425
4521
|
} catch {
|
|
@@ -14286,6 +14382,41 @@ __factories["./src/mcp/handlers"] = function(module, exports) {
|
|
|
14286
14382
|
return out.join('\n');
|
|
14287
14383
|
}
|
|
14288
14384
|
|
|
14385
|
+
/**
|
|
14386
|
+
* get_budget({ session?, budgetTokens? }) → string
|
|
14387
|
+
*
|
|
14388
|
+
* Session spend ledger (F1): estimated tokens SigMap emitted this session,
|
|
14389
|
+
* optional budget remaining, and context freshness. Estimates only (chars/4);
|
|
14390
|
+
* does NOT see the host chat's total spend.
|
|
14391
|
+
*/
|
|
14392
|
+
function getBudget(args, cwd) {
|
|
14393
|
+
const { budgetStatus } = __require('./src/tracking/budget');
|
|
14394
|
+
let config = {};
|
|
14395
|
+
try { config = __require('./src/config/loader').loadConfig(cwd) || {}; } catch (_) {}
|
|
14396
|
+
const opts = { config };
|
|
14397
|
+
if (args && args.session) opts.session = String(args.session);
|
|
14398
|
+
if (args && args.budgetTokens != null) opts.budgetTokens = Number(args.budgetTokens);
|
|
14399
|
+
const s = budgetStatus(cwd, opts);
|
|
14400
|
+
|
|
14401
|
+
const out = ['# SigMap session spend (estimates — chars/4; SigMap-emitted tokens only)'];
|
|
14402
|
+
out.push('');
|
|
14403
|
+
out.push(`Session : ${s.session}`);
|
|
14404
|
+
out.push(`Ops : ${s.ops}`);
|
|
14405
|
+
out.push(`Spent : ~${s.spentTokens.toLocaleString()} tokens (baseline ~${s.baselineTokens.toLocaleString()}, saved ~${s.savedTokens.toLocaleString()})`);
|
|
14406
|
+
if (s.budgetTokens != null) {
|
|
14407
|
+
out.push(`Budget : ${s.budgetTokens.toLocaleString()} → remaining ~${s.remainingTokens.toLocaleString()} (${s.pctUsed}% used)${s.overBudget ? ' ⚠ OVER BUDGET' : ''}`);
|
|
14408
|
+
if (s.overBudget || (s.pctUsed != null && s.pctUsed >= 80)) {
|
|
14409
|
+
out.push('Advice : prefer terse output, squeeze large inputs, summarize-then-drop context.');
|
|
14410
|
+
}
|
|
14411
|
+
} else {
|
|
14412
|
+
out.push('Budget : none set (config sessionBudgetTokens or budgetTokens arg)');
|
|
14413
|
+
}
|
|
14414
|
+
out.push(s.context.exists
|
|
14415
|
+
? `Context : ${s.context.ageDays} day(s) old${s.context.stale ? ` — STALE (> ${s.context.ttlDays}d TTL); re-run sigmap` : ''}`
|
|
14416
|
+
: 'Context : no generated context found — run sigmap first');
|
|
14417
|
+
return out.join('\n');
|
|
14418
|
+
}
|
|
14419
|
+
|
|
14289
14420
|
/**
|
|
14290
14421
|
* get_callee_signatures — return the exact defining signature(s) of named
|
|
14291
14422
|
* symbols from the index, so an agent never guesses a callee's parameter types.
|
|
@@ -14679,7 +14810,7 @@ __factories["./src/mcp/handlers"] = function(module, exports) {
|
|
|
14679
14810
|
return header + sq.squeezed;
|
|
14680
14811
|
}
|
|
14681
14812
|
|
|
14682
|
-
module.exports = { readContext, searchSignatures, getMap, createCheckpoint, getRouting, explainFile, listModules, queryContext, getMethodImpact, getImpact, getLines, readMemory, getCalleeSignatures, notifyFileCreated, notifySymbolAdded, notifyFileDeleted, getDiffContext, getArchitectureOverview, verifySuggestion, squeezeOutput };
|
|
14813
|
+
module.exports = { readContext, searchSignatures, getMap, createCheckpoint, getRouting, explainFile, listModules, queryContext, getMethodImpact, getImpact, getLines, readMemory, getCalleeSignatures, notifyFileCreated, notifySymbolAdded, notifyFileDeleted, getDiffContext, getArchitectureOverview, verifySuggestion, squeezeOutput, getBudget };
|
|
14683
14814
|
|
|
14684
14815
|
};
|
|
14685
14816
|
|
|
@@ -14846,11 +14977,11 @@ __factories["./src/mcp/server"] = function(module, exports) {
|
|
|
14846
14977
|
|
|
14847
14978
|
const readline = require('readline');
|
|
14848
14979
|
const { TOOLS } = __require('./src/mcp/tools');
|
|
14849
|
-
const { readContext, searchSignatures, getMap, createCheckpoint, getRouting, explainFile, listModules, queryContext, getMethodImpact, getImpact, getLines, readMemory, getCalleeSignatures, notifyFileCreated, notifySymbolAdded, notifyFileDeleted, getDiffContext, getArchitectureOverview, verifySuggestion, squeezeOutput } = __require('./src/mcp/handlers');
|
|
14980
|
+
const { readContext, searchSignatures, getMap, createCheckpoint, getRouting, explainFile, listModules, queryContext, getMethodImpact, getImpact, getLines, readMemory, getCalleeSignatures, notifyFileCreated, notifySymbolAdded, notifyFileDeleted, getDiffContext, getArchitectureOverview, verifySuggestion, squeezeOutput, getBudget } = __require('./src/mcp/handlers');
|
|
14850
14981
|
|
|
14851
14982
|
const SERVER_INFO = {
|
|
14852
14983
|
name: 'sigmap',
|
|
14853
|
-
version: '8.
|
|
14984
|
+
version: '8.23.0',
|
|
14854
14985
|
description: 'SigMap MCP server — code signatures on demand',
|
|
14855
14986
|
};
|
|
14856
14987
|
|
|
@@ -14918,6 +15049,7 @@ __factories["./src/mcp/server"] = function(module, exports) {
|
|
|
14918
15049
|
else if (name === 'get_architecture_overview') text = getArchitectureOverview(args, cwd);
|
|
14919
15050
|
else if (name === 'verify_suggestion') text = verifySuggestion(args, cwd);
|
|
14920
15051
|
else if (name === 'squeeze_output') text = squeezeOutput(args, cwd);
|
|
15052
|
+
else if (name === 'get_budget') text = getBudget(args, cwd);
|
|
14921
15053
|
else {
|
|
14922
15054
|
respondError(id, -32601, `Unknown tool: ${name}`);
|
|
14923
15055
|
return;
|
|
@@ -15364,6 +15496,32 @@ __factories["./src/mcp/tools"] = function(module, exports) {
|
|
|
15364
15496
|
required: ['content'],
|
|
15365
15497
|
},
|
|
15366
15498
|
},
|
|
15499
|
+
{
|
|
15500
|
+
name: 'get_budget',
|
|
15501
|
+
description:
|
|
15502
|
+
'Session spend ledger — estimated tokens SigMap has emitted this session ' +
|
|
15503
|
+
'(chars/4 estimates from the local gain log), optional budget remaining, and ' +
|
|
15504
|
+
'context freshness. Scope honesty: counts only SigMap output, NOT the whole ' +
|
|
15505
|
+
"chat's spend. Check this mid-session and degrade gracefully when near budget: " +
|
|
15506
|
+
'prefer terse encoding, squeeze large outputs, summarize-then-drop context. ' +
|
|
15507
|
+
'Local-only; no LLM, no network.',
|
|
15508
|
+
inputSchema: {
|
|
15509
|
+
type: 'object',
|
|
15510
|
+
properties: {
|
|
15511
|
+
session: {
|
|
15512
|
+
type: 'string',
|
|
15513
|
+
description:
|
|
15514
|
+
'Session key to report on (default: SIGMAP_SESSION env or the UTC day bucket).',
|
|
15515
|
+
},
|
|
15516
|
+
budgetTokens: {
|
|
15517
|
+
type: 'number',
|
|
15518
|
+
description:
|
|
15519
|
+
'Budget override in estimated tokens (default: config sessionBudgetTokens).',
|
|
15520
|
+
},
|
|
15521
|
+
},
|
|
15522
|
+
required: [],
|
|
15523
|
+
},
|
|
15524
|
+
},
|
|
15367
15525
|
];
|
|
15368
15526
|
|
|
15369
15527
|
module.exports = { TOOLS };
|
|
@@ -18320,6 +18478,123 @@ __factories["./src/tracking/aggregate"] = function(module, exports) {
|
|
|
18320
18478
|
|
|
18321
18479
|
};
|
|
18322
18480
|
|
|
18481
|
+
// ── ./src/tracking/budget ──
|
|
18482
|
+
__factories["./src/tracking/budget"] = function(module, exports) {
|
|
18483
|
+
|
|
18484
|
+
/**
|
|
18485
|
+
* Session spend ledger (v8.23 F1) — a queryable view over the existing gain
|
|
18486
|
+
* log (.context/gain.ndjson, written by recordUsage).
|
|
18487
|
+
*
|
|
18488
|
+
* Honesty scope: this ledger counts tokens **SigMap emitted** (chars/4
|
|
18489
|
+
* estimates), not the host chat's total spend — conversation history, model
|
|
18490
|
+
* output, and other tools are invisible to a CLI. Every number here is an
|
|
18491
|
+
* estimate and is labeled as such in output.
|
|
18492
|
+
*
|
|
18493
|
+
* Session identity: `SIGMAP_SESSION` env var when the host sets one, else a
|
|
18494
|
+
* UTC day bucket (YYYY-MM-DD). Legacy gain entries (written before the
|
|
18495
|
+
* `session` field existed) match day-bucket sessions by timestamp prefix.
|
|
18496
|
+
*
|
|
18497
|
+
* Zero dependencies; local JSON only.
|
|
18498
|
+
*/
|
|
18499
|
+
|
|
18500
|
+
const fs = require('fs');
|
|
18501
|
+
const path = require('path');
|
|
18502
|
+
const { readGainLog } = __require('./src/tracking/logger');
|
|
18503
|
+
|
|
18504
|
+
// Same generated-context surfaces cache/freshen.js watches.
|
|
18505
|
+
const CONTEXT_PATHS = [
|
|
18506
|
+
['.github', 'copilot-instructions.md'],
|
|
18507
|
+
['CLAUDE.md'], ['AGENTS.md'], ['.github', 'context-cold.md'],
|
|
18508
|
+
];
|
|
18509
|
+
|
|
18510
|
+
/** Session key: SIGMAP_SESSION override, else UTC day bucket. */
|
|
18511
|
+
function sessionKey(env) {
|
|
18512
|
+
const e = env || process.env;
|
|
18513
|
+
if (e.SIGMAP_SESSION) return String(e.SIGMAP_SESSION);
|
|
18514
|
+
return new Date().toISOString().slice(0, 10);
|
|
18515
|
+
}
|
|
18516
|
+
|
|
18517
|
+
/** Newest mtime (ms) among generated context files, or 0 if none exist. */
|
|
18518
|
+
function contextMtime(cwd) {
|
|
18519
|
+
let newest = 0;
|
|
18520
|
+
for (const parts of CONTEXT_PATHS) {
|
|
18521
|
+
try { newest = Math.max(newest, fs.statSync(path.join(cwd, ...parts)).mtimeMs); } catch (_) {}
|
|
18522
|
+
}
|
|
18523
|
+
return newest;
|
|
18524
|
+
}
|
|
18525
|
+
|
|
18526
|
+
/** Does a gain-log entry belong to this session? */
|
|
18527
|
+
function entryInSession(entry, session) {
|
|
18528
|
+
if (entry.session) return entry.session === session;
|
|
18529
|
+
// Legacy entry: match day-bucket sessions on the timestamp date.
|
|
18530
|
+
return /^\d{4}-\d{2}-\d{2}$/.test(session) && String(entry.ts || '').startsWith(session);
|
|
18531
|
+
}
|
|
18532
|
+
|
|
18533
|
+
/**
|
|
18534
|
+
* Session spend status.
|
|
18535
|
+
* @param {string} cwd
|
|
18536
|
+
* @param {object} [opts]
|
|
18537
|
+
* @param {string} [opts.session] session key (default: sessionKey())
|
|
18538
|
+
* @param {number} [opts.budgetTokens] budget override (else config sessionBudgetTokens)
|
|
18539
|
+
* @param {number} [opts.contextTtlDays] TTL override (else config contextTtlDays)
|
|
18540
|
+
* @param {object} [opts.config] loaded config (for the two keys above)
|
|
18541
|
+
* @param {number} [opts.now] clock override for tests (ms)
|
|
18542
|
+
* @returns {{
|
|
18543
|
+
* session: string, unit: 'estimated-tokens', ops: number,
|
|
18544
|
+
* spentTokens: number, baselineTokens: number, savedTokens: number,
|
|
18545
|
+
* budgetTokens: number|null, remainingTokens: number|null, pctUsed: number|null,
|
|
18546
|
+
* overBudget: boolean,
|
|
18547
|
+
* context: { exists: boolean, ageMs: number|null, ageDays: number|null,
|
|
18548
|
+
* ttlDays: number|null, stale: boolean }
|
|
18549
|
+
* }}
|
|
18550
|
+
*/
|
|
18551
|
+
function budgetStatus(cwd, opts = {}) {
|
|
18552
|
+
const session = opts.session || sessionKey();
|
|
18553
|
+
const cfg = opts.config || {};
|
|
18554
|
+
const budget = opts.budgetTokens != null ? Number(opts.budgetTokens)
|
|
18555
|
+
: (Number.isFinite(cfg.sessionBudgetTokens) ? cfg.sessionBudgetTokens : null);
|
|
18556
|
+
const ttlDays = opts.contextTtlDays != null ? Number(opts.contextTtlDays)
|
|
18557
|
+
: (Number.isFinite(cfg.contextTtlDays) ? cfg.contextTtlDays : null);
|
|
18558
|
+
const now = opts.now != null ? opts.now : Date.now();
|
|
18559
|
+
|
|
18560
|
+
let ops = 0, spent = 0, baseline = 0, saved = 0;
|
|
18561
|
+
for (const e of readGainLog(cwd)) {
|
|
18562
|
+
if (!entryInSession(e, session)) continue;
|
|
18563
|
+
ops++;
|
|
18564
|
+
spent += Number(e.actualTokens) || 0;
|
|
18565
|
+
baseline += Number(e.baselineTokens) || 0;
|
|
18566
|
+
saved += Number(e.savedTokens) || 0;
|
|
18567
|
+
}
|
|
18568
|
+
|
|
18569
|
+
const mtime = contextMtime(cwd);
|
|
18570
|
+
const ageMs = mtime > 0 ? Math.max(0, now - mtime) : null;
|
|
18571
|
+
const ageDays = ageMs != null ? ageMs / 86400000 : null;
|
|
18572
|
+
|
|
18573
|
+
return {
|
|
18574
|
+
session,
|
|
18575
|
+
unit: 'estimated-tokens',
|
|
18576
|
+
ops,
|
|
18577
|
+
spentTokens: spent,
|
|
18578
|
+
baselineTokens: baseline,
|
|
18579
|
+
savedTokens: saved,
|
|
18580
|
+
budgetTokens: budget,
|
|
18581
|
+
remainingTokens: budget != null ? Math.max(0, budget - spent) : null,
|
|
18582
|
+
pctUsed: budget > 0 ? Math.round((spent / budget) * 1000) / 10 : null,
|
|
18583
|
+
overBudget: budget != null && spent > budget,
|
|
18584
|
+
context: {
|
|
18585
|
+
exists: mtime > 0,
|
|
18586
|
+
ageMs,
|
|
18587
|
+
ageDays: ageDays != null ? Math.round(ageDays * 10) / 10 : null,
|
|
18588
|
+
ttlDays,
|
|
18589
|
+
stale: ttlDays != null && ageDays != null && ageDays > ttlDays,
|
|
18590
|
+
},
|
|
18591
|
+
};
|
|
18592
|
+
}
|
|
18593
|
+
|
|
18594
|
+
module.exports = { sessionKey, budgetStatus, contextMtime, entryInSession };
|
|
18595
|
+
|
|
18596
|
+
};
|
|
18597
|
+
|
|
18323
18598
|
// ── ./src/tracking/logger ──
|
|
18324
18599
|
__factories["./src/tracking/logger"] = function(module, exports) {
|
|
18325
18600
|
|
|
@@ -18498,10 +18773,12 @@ __factories["./src/tracking/logger"] = function(module, exports) {
|
|
|
18498
18773
|
const baseline = Math.max(0, Number(entry.baselineTokens) || 0);
|
|
18499
18774
|
const actual = Math.max(0, Number(entry.actualTokens) || 0);
|
|
18500
18775
|
const saved = Math.max(0, baseline - actual);
|
|
18776
|
+
const ts = new Date().toISOString();
|
|
18501
18777
|
const record = {
|
|
18502
|
-
ts
|
|
18778
|
+
ts,
|
|
18503
18779
|
v: entry.version || '0.9.0',
|
|
18504
18780
|
op: entry.op || 'generate',
|
|
18781
|
+
session: entry.session || process.env.SIGMAP_SESSION || ts.slice(0, 10),
|
|
18505
18782
|
baselineTokens: baseline,
|
|
18506
18783
|
actualTokens: actual,
|
|
18507
18784
|
savedTokens: saved,
|
|
@@ -18544,6 +18821,8 @@ __factories["./src/tracking/pricing"] = function(module, exports) {
|
|
|
18544
18821
|
'gpt-4o-mini': 0.15,
|
|
18545
18822
|
'gemini-1.5-pro': 1.25,
|
|
18546
18823
|
'gemini-1.5-flash': 0.075,
|
|
18824
|
+
'minimax-m3': 0.6,
|
|
18825
|
+
'minimax-m2.7': 0.3,
|
|
18547
18826
|
};
|
|
18548
18827
|
|
|
18549
18828
|
const DEFAULT_MODEL = 'claude-sonnet';
|
|
@@ -20068,7 +20347,7 @@ function __tryGit(args, opts = {}) {
|
|
|
20068
20347
|
catch (_) { return ''; }
|
|
20069
20348
|
}
|
|
20070
20349
|
|
|
20071
|
-
const VERSION = '8.
|
|
20350
|
+
const VERSION = '8.23.0';
|
|
20072
20351
|
const MARKER = '\n\n## Auto-generated signatures\n<!-- Updated by gen-context.js -->\n';
|
|
20073
20352
|
|
|
20074
20353
|
function requireSourceOrBundled(key) {
|
|
@@ -21953,6 +22232,8 @@ Usage:
|
|
|
21953
22232
|
${cmd} evidence "<query>" --top <n> --budget <n> --out <path> Tune ranked files / token budget / write rendered output
|
|
21954
22233
|
${cmd} memory List cross-session stores (.context/) — entries, size, age
|
|
21955
22234
|
${cmd} memory --clear <store> Clear one store: session|notes|weights|evidence|all (--json supported)
|
|
22235
|
+
${cmd} budget Session spend ledger — estimated SigMap-emitted tokens, budget, context age (--json)
|
|
22236
|
+
${cmd} budget --budget <tokens> One-off budget override (config: sessionBudgetTokens, contextTtlDays)
|
|
21956
22237
|
${cmd} note "<text>" Append a note to the cross-session decision log
|
|
21957
22238
|
${cmd} note List recent notes (also: note --list <N>)
|
|
21958
22239
|
${cmd} status Show repo state — branch, dirty files, index freshness, notes
|
|
@@ -23404,6 +23685,36 @@ function main() {
|
|
|
23404
23685
|
process.exit(0);
|
|
23405
23686
|
}
|
|
23406
23687
|
|
|
23688
|
+
if (args[0] === 'budget') {
|
|
23689
|
+
const jsonOut = args.includes('--json');
|
|
23690
|
+
const { budgetStatus } = requireSourceOrBundled('./src/tracking/budget');
|
|
23691
|
+
let config = {};
|
|
23692
|
+
try { config = requireSourceOrBundled('./src/config/loader').loadConfig(cwd) || {}; } catch (_) {}
|
|
23693
|
+
const opts = { config };
|
|
23694
|
+
const sIdx = args.indexOf('--session');
|
|
23695
|
+
if (sIdx !== -1 && args[sIdx + 1] && !args[sIdx + 1].startsWith('--')) opts.session = args[sIdx + 1];
|
|
23696
|
+
const bIdx = args.indexOf('--budget');
|
|
23697
|
+
if (bIdx !== -1 && args[bIdx + 1] && !args[bIdx + 1].startsWith('--')) opts.budgetTokens = Number(args[bIdx + 1]);
|
|
23698
|
+
const st = budgetStatus(cwd, opts);
|
|
23699
|
+
if (jsonOut) {
|
|
23700
|
+
process.stdout.write(JSON.stringify(st) + '\n');
|
|
23701
|
+
process.exit(0);
|
|
23702
|
+
}
|
|
23703
|
+
console.log('[sigmap] session spend (estimates — chars/4; SigMap-emitted tokens only)');
|
|
23704
|
+
console.log(` session ${st.session}`);
|
|
23705
|
+
console.log(` ops ${st.ops}`);
|
|
23706
|
+
console.log(` spent ~${st.spentTokens.toLocaleString()} tokens (baseline ~${st.baselineTokens.toLocaleString()}, saved ~${st.savedTokens.toLocaleString()})`);
|
|
23707
|
+
if (st.budgetTokens != null) {
|
|
23708
|
+
console.log(` budget ${st.budgetTokens.toLocaleString()} → remaining ~${st.remainingTokens.toLocaleString()} (${st.pctUsed}% used)${st.overBudget ? ' ⚠ OVER BUDGET' : ''}`);
|
|
23709
|
+
} else {
|
|
23710
|
+
console.log(' budget none set (config sessionBudgetTokens or --budget <tokens>)');
|
|
23711
|
+
}
|
|
23712
|
+
console.log(st.context.exists
|
|
23713
|
+
? ` context ${st.context.ageDays} day(s) old${st.context.stale ? ` ⚠ STALE (> ${st.context.ttlDays}d TTL) — re-run sigmap` : ''}`
|
|
23714
|
+
: ' context none generated yet — run sigmap first');
|
|
23715
|
+
process.exit(0);
|
|
23716
|
+
}
|
|
23717
|
+
|
|
23407
23718
|
if (args[0] === 'note') {
|
|
23408
23719
|
const jsonOut = args.includes('--json');
|
|
23409
23720
|
const { addNote, readNotes, formatNotes } = requireSourceOrBundled('./src/session/notes');
|
package/llms-full.txt
CHANGED
|
@@ -11,22 +11,22 @@ ranking keeps the relevant context in scope (cutting tokens ~97% as a side
|
|
|
11
11
|
effect), with no LLM calls, embeddings, or vector database. Works with Claude,
|
|
12
12
|
Cursor, GitHub Copilot, Aider, Windsurf, local LLMs, and MCP.
|
|
13
13
|
|
|
14
|
-
# Version: 8.
|
|
14
|
+
# Version: 8.23.0 | Benchmark: sigmap-v8.23-main (2026-07-28)
|
|
15
15
|
# Source: auto-generated from package.json, version.json, benchmarks/latest.json, src/mcp/tools.js, src/config/defaults.js
|
|
16
16
|
# Regenerate: npm run generate:llms | Validate: npm run validate:llms
|
|
17
17
|
|
|
18
18
|
---
|
|
19
19
|
|
|
20
|
-
## Core metrics (benchmark: sigmap-v8.
|
|
20
|
+
## Core metrics (benchmark: sigmap-v8.23-main, 2026-07-28)
|
|
21
21
|
|
|
22
22
|
| Metric | Without SigMap | With SigMap |
|
|
23
23
|
|--------|----------------|-------------|
|
|
24
|
-
| Retrieval hit@5 |
|
|
24
|
+
| Retrieval hit@5 | 44.8% (single-shot grep) | 82.2% (1.59× lift) |
|
|
25
25
|
| Token reduction | — | 96.8% average |
|
|
26
|
-
| Task-success proxy (modeled) | — |
|
|
27
|
-
| Prompts per task | 2.84 | 1.
|
|
26
|
+
| Task-success proxy (modeled) | — | 64.8% |
|
|
27
|
+
| Prompts per task | 2.84 | 1.53 (46.1% fewer) |
|
|
28
28
|
| Supported languages | — | 33 |
|
|
29
|
-
| MCP tools | — |
|
|
29
|
+
| MCP tools | — | 21 |
|
|
30
30
|
| npm runtime dependencies | — | 0 |
|
|
31
31
|
|
|
32
32
|
---
|
|
@@ -125,6 +125,8 @@ sigmap evidence "<query>" --markdown Emit the Markdown handoff rendering to
|
|
|
125
125
|
sigmap evidence "<query>" --top <n> --budget <n> --out <path> Tune ranked files / token budget / write rendered output
|
|
126
126
|
sigmap memory List cross-session stores (.context/) — entries, size, age
|
|
127
127
|
sigmap memory --clear <store> Clear one store: session|notes|weights|evidence|all (--json supported)
|
|
128
|
+
sigmap budget Session spend ledger — estimated SigMap-emitted tokens, budget, context age (--json)
|
|
129
|
+
sigmap budget --budget <tokens> One-off budget override (config: sessionBudgetTokens, contextTtlDays)
|
|
128
130
|
sigmap note "<text>" Append a note to the cross-session decision log
|
|
129
131
|
sigmap note List recent notes (also: note --list <N>)
|
|
130
132
|
sigmap status Show repo state — branch, dirty files, index freshness, notes
|
|
@@ -138,7 +140,7 @@ sigmap --version Show version
|
|
|
138
140
|
|
|
139
141
|
---
|
|
140
142
|
|
|
141
|
-
## MCP server —
|
|
143
|
+
## MCP server — 21 tools
|
|
142
144
|
|
|
143
145
|
Start with `sigmap --mcp` (stdio JSON-RPC). Configure once:
|
|
144
146
|
|
|
@@ -306,6 +308,14 @@ Compress noisy tool, command, or agent output before it enters context — a sta
|
|
|
306
308
|
Input: { content: string }
|
|
307
309
|
```
|
|
308
310
|
|
|
311
|
+
### get_budget
|
|
312
|
+
|
|
313
|
+
Session spend ledger — estimated tokens SigMap has emitted this session (chars/4 estimates from the local gain log), optional budget remaining, and context freshness. Scope honesty: counts only SigMap output, NOT the whole chat's spend. Check this mid-session and degrade gracefully when near budget: prefer terse encoding, squeeze large outputs, summarize-then-drop context. Local-only; no LLM, no network.
|
|
314
|
+
|
|
315
|
+
```
|
|
316
|
+
Input: { session?: string, budgetTokens?: number }
|
|
317
|
+
```
|
|
318
|
+
|
|
309
319
|
---
|
|
310
320
|
|
|
311
321
|
## Configuration (gen-context.config.json)
|
|
@@ -334,6 +344,8 @@ watchDebounce = 300
|
|
|
334
344
|
routing = false
|
|
335
345
|
format = default
|
|
336
346
|
tracking = false
|
|
347
|
+
sessionBudgetTokens = null
|
|
348
|
+
contextTtlDays = null
|
|
337
349
|
mcp = {"autoRegister":true}
|
|
338
350
|
depMap = true
|
|
339
351
|
versionPins = true
|
package/llms.txt
CHANGED
|
@@ -11,7 +11,7 @@ ranking keeps the relevant context in scope (cutting tokens ~97% as a side
|
|
|
11
11
|
effect), with no LLM calls, embeddings, or vector database. Works with Claude,
|
|
12
12
|
Cursor, GitHub Copilot, Aider, Windsurf, local LLMs, and MCP.
|
|
13
13
|
|
|
14
|
-
# Version: 8.
|
|
14
|
+
# Version: 8.23.0 | Benchmark: sigmap-v8.23-main (2026-07-28)
|
|
15
15
|
# Source: auto-generated from package.json, version.json, benchmarks/latest.json, src/mcp/tools.js, src/config/defaults.js
|
|
16
16
|
# Regenerate: npm run generate:llms | Validate: npm run validate:llms
|
|
17
17
|
|
|
@@ -23,13 +23,13 @@ Cursor, GitHub Copilot, Aider, Windsurf, local LLMs, and MCP.
|
|
|
23
23
|
- No blast-radius awareness before editing a hub file — `--impact` shows every file a change touches.
|
|
24
24
|
- Pasted stack traces, CI logs, and JSON bloat the prompt — `squeeze` minimizes them and enriches the top frame from the symbol index.
|
|
25
25
|
|
|
26
|
-
## Core metrics (benchmark: sigmap-v8.
|
|
26
|
+
## Core metrics (benchmark: sigmap-v8.23-main, 2026-07-28)
|
|
27
27
|
|
|
28
|
-
- hit@5 retrieval:
|
|
28
|
+
- hit@5 retrieval: 82.2% vs 44.8% single-shot grep baseline (1.59× lift)
|
|
29
29
|
- Token reduction: 96.8% average across benchmark repos
|
|
30
|
-
- Task-success proxy:
|
|
31
|
-
- Prompts per task: 1.
|
|
32
|
-
- Languages: 33 supported · MCP tools:
|
|
30
|
+
- Task-success proxy: 64.8% (modeled from retrieval tiers, not measured LLM sessions)
|
|
31
|
+
- Prompts per task: 1.53 vs 2.84 baseline (46.1% fewer, modeled)
|
|
32
|
+
- Languages: 33 supported · MCP tools: 21
|
|
33
33
|
- Dependencies: zero npm runtime dependencies · fully offline
|
|
34
34
|
|
|
35
35
|
## Quick start
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "sigmap",
|
|
3
|
-
"version": "8.
|
|
3
|
+
"version": "8.23.0",
|
|
4
4
|
"description": "The deterministic, verifiable grounding layer for AI code work — a zero-dependency signature-and-evidence map that grounds Claude, Cursor, Copilot, Aider, Windsurf, local LLMs & MCP agents against your real code (repo + installed libraries) so they stop hallucinating files, imports & APIs. Runs offline via npx; byte-stable output; ~97% token reduction as proof.",
|
|
5
5
|
"main": "packages/core/index.js",
|
|
6
6
|
"exports": {
|
package/src/config/defaults.js
CHANGED
|
@@ -106,6 +106,14 @@ const DEFAULTS = {
|
|
|
106
106
|
// Append run metrics to .context/usage.ndjson after each generate
|
|
107
107
|
tracking: false,
|
|
108
108
|
|
|
109
|
+
// Session spend ledger (`sigmap budget` / MCP get_budget). Estimates only —
|
|
110
|
+
// counts tokens SigMap emitted (chars/4), not the host chat's total spend.
|
|
111
|
+
// Number → warn threshold for estimated SigMap-emitted tokens per session.
|
|
112
|
+
sessionBudgetTokens: null,
|
|
113
|
+
|
|
114
|
+
// Number of days before generated context counts as stale in budget output.
|
|
115
|
+
contextTtlDays: null,
|
|
116
|
+
|
|
109
117
|
// MCP server configuration
|
|
110
118
|
mcp: {
|
|
111
119
|
autoRegister: true,
|
|
@@ -0,0 +1,83 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
|
|
3
|
+
/**
|
|
4
|
+
* Task-corpus hygiene (A3, v8.22 "Hard Corpus").
|
|
5
|
+
*
|
|
6
|
+
* A benchmark query "leaks" when it shares a token with the basenames of its
|
|
7
|
+
* expected files — hit@5 then partly measures filename matching, not
|
|
8
|
+
* retrieval. The criterion is deterministic and reuses the production
|
|
9
|
+
* tokenizer (identifier splitting + stemming from src/retrieval/bm25.js), so
|
|
10
|
+
* "payments" leaks against payment.js and "InterceptorManager" leaks against
|
|
11
|
+
* "interceptor manager" the same way the ranker would see them.
|
|
12
|
+
*
|
|
13
|
+
* Tasks carry an optional `split` field: 'hard' tasks MUST be leak-free
|
|
14
|
+
* (validateTasks reports them as violations); 'easy' tasks (the default) may
|
|
15
|
+
* leak — that is what makes them easy.
|
|
16
|
+
*
|
|
17
|
+
* Size buckets group repos by indexed file count so large repos stop being
|
|
18
|
+
* averaged away by tiny ones. Thresholds are the rough tertiles of the
|
|
19
|
+
* current benchmarks/repos corpus (43 repos, 27–3450 source files).
|
|
20
|
+
*/
|
|
21
|
+
|
|
22
|
+
const { tokenize } = require('../retrieval/bm25');
|
|
23
|
+
|
|
24
|
+
const BUCKET_LIMITS = { small: 200, medium: 1000 }; // files; large = above medium
|
|
25
|
+
|
|
26
|
+
/**
|
|
27
|
+
* Stemmed tokens of a file path's basename (extension stripped).
|
|
28
|
+
* @param {string} filePath
|
|
29
|
+
* @returns {string[]}
|
|
30
|
+
*/
|
|
31
|
+
function basenameTokens(filePath) {
|
|
32
|
+
const base = String(filePath).split('/').pop() || '';
|
|
33
|
+
return tokenize(base.replace(/\.[^.]*$/, ''));
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
/**
|
|
37
|
+
* Leaked tokens between a query and its expected files' basenames.
|
|
38
|
+
* @param {string} query
|
|
39
|
+
* @param {string[]} expectedFiles
|
|
40
|
+
* @returns {{ leaked: string[], clean: boolean }}
|
|
41
|
+
*/
|
|
42
|
+
function queryLeakage(query, expectedFiles) {
|
|
43
|
+
const qToks = new Set(tokenize(query));
|
|
44
|
+
const leaked = new Set();
|
|
45
|
+
for (const f of expectedFiles || []) {
|
|
46
|
+
for (const t of basenameTokens(f)) {
|
|
47
|
+
if (qToks.has(t)) leaked.add(t);
|
|
48
|
+
}
|
|
49
|
+
}
|
|
50
|
+
return { leaked: [...leaked].sort(), clean: leaked.size === 0 };
|
|
51
|
+
}
|
|
52
|
+
|
|
53
|
+
/**
|
|
54
|
+
* Validate a task list: every task gets a leakage result; hard-split tasks
|
|
55
|
+
* that leak are violations.
|
|
56
|
+
* @param {Array<{id?:string, query:string, expected_files?:string[], split?:string}>} tasks
|
|
57
|
+
* @returns {{ results: object[], hardViolations: object[] }}
|
|
58
|
+
*/
|
|
59
|
+
function validateTasks(tasks) {
|
|
60
|
+
const results = [];
|
|
61
|
+
const hardViolations = [];
|
|
62
|
+
for (const t of tasks || []) {
|
|
63
|
+
const split = t.split === 'hard' ? 'hard' : 'easy';
|
|
64
|
+
const { leaked, clean } = queryLeakage(t.query, t.expected_files);
|
|
65
|
+
const row = { id: t.id || '?', split, leaked, clean };
|
|
66
|
+
results.push(row);
|
|
67
|
+
if (split === 'hard' && !clean) hardViolations.push(row);
|
|
68
|
+
}
|
|
69
|
+
return { results, hardViolations };
|
|
70
|
+
}
|
|
71
|
+
|
|
72
|
+
/**
|
|
73
|
+
* Size bucket for a repo by indexed file count.
|
|
74
|
+
* @param {number} fileCount
|
|
75
|
+
* @returns {'small'|'medium'|'large'}
|
|
76
|
+
*/
|
|
77
|
+
function sizeBucket(fileCount) {
|
|
78
|
+
if (fileCount < BUCKET_LIMITS.small) return 'small';
|
|
79
|
+
if (fileCount <= BUCKET_LIMITS.medium) return 'medium';
|
|
80
|
+
return 'large';
|
|
81
|
+
}
|
|
82
|
+
|
|
83
|
+
module.exports = { basenameTokens, queryLeakage, validateTasks, sizeBucket, BUCKET_LIMITS };
|
package/src/eval/runner.js
CHANGED
|
@@ -124,7 +124,7 @@ function estimateTokens(sigs) {
|
|
|
124
124
|
|
|
125
125
|
/**
|
|
126
126
|
* Load tasks from a JSONL file.
|
|
127
|
-
* Each line: { id, query, expected_files, repo }
|
|
127
|
+
* Each line: { id, query, expected_files, repo, split? ('easy'|'hard') }
|
|
128
128
|
* Invalid or blank lines are silently skipped.
|
|
129
129
|
* @param {string} tasksFile - absolute or relative path
|
|
130
130
|
* @returns {Array<{id:string, query:string, expected:string[], repo:string}>}
|
|
@@ -144,6 +144,7 @@ function loadTasks(tasksFile) {
|
|
|
144
144
|
query: obj.query,
|
|
145
145
|
expected: obj.expected_files,
|
|
146
146
|
repo: obj.repo || '.',
|
|
147
|
+
split: obj.split === 'hard' ? 'hard' : 'easy',
|
|
147
148
|
});
|
|
148
149
|
}
|
|
149
150
|
} catch {
|
package/src/mcp/handlers.js
CHANGED
|
@@ -586,6 +586,41 @@ function readMemory(args, cwd) {
|
|
|
586
586
|
return out.join('\n');
|
|
587
587
|
}
|
|
588
588
|
|
|
589
|
+
/**
|
|
590
|
+
* get_budget({ session?, budgetTokens? }) → string
|
|
591
|
+
*
|
|
592
|
+
* Session spend ledger (F1): estimated tokens SigMap emitted this session,
|
|
593
|
+
* optional budget remaining, and context freshness. Estimates only (chars/4);
|
|
594
|
+
* does NOT see the host chat's total spend.
|
|
595
|
+
*/
|
|
596
|
+
function getBudget(args, cwd) {
|
|
597
|
+
const { budgetStatus } = require('../tracking/budget');
|
|
598
|
+
let config = {};
|
|
599
|
+
try { config = require('../config/loader').loadConfig(cwd) || {}; } catch (_) {}
|
|
600
|
+
const opts = { config };
|
|
601
|
+
if (args && args.session) opts.session = String(args.session);
|
|
602
|
+
if (args && args.budgetTokens != null) opts.budgetTokens = Number(args.budgetTokens);
|
|
603
|
+
const s = budgetStatus(cwd, opts);
|
|
604
|
+
|
|
605
|
+
const out = ['# SigMap session spend (estimates — chars/4; SigMap-emitted tokens only)'];
|
|
606
|
+
out.push('');
|
|
607
|
+
out.push(`Session : ${s.session}`);
|
|
608
|
+
out.push(`Ops : ${s.ops}`);
|
|
609
|
+
out.push(`Spent : ~${s.spentTokens.toLocaleString()} tokens (baseline ~${s.baselineTokens.toLocaleString()}, saved ~${s.savedTokens.toLocaleString()})`);
|
|
610
|
+
if (s.budgetTokens != null) {
|
|
611
|
+
out.push(`Budget : ${s.budgetTokens.toLocaleString()} → remaining ~${s.remainingTokens.toLocaleString()} (${s.pctUsed}% used)${s.overBudget ? ' ⚠ OVER BUDGET' : ''}`);
|
|
612
|
+
if (s.overBudget || (s.pctUsed != null && s.pctUsed >= 80)) {
|
|
613
|
+
out.push('Advice : prefer terse output, squeeze large inputs, summarize-then-drop context.');
|
|
614
|
+
}
|
|
615
|
+
} else {
|
|
616
|
+
out.push('Budget : none set (config sessionBudgetTokens or budgetTokens arg)');
|
|
617
|
+
}
|
|
618
|
+
out.push(s.context.exists
|
|
619
|
+
? `Context : ${s.context.ageDays} day(s) old${s.context.stale ? ` — STALE (> ${s.context.ttlDays}d TTL); re-run sigmap` : ''}`
|
|
620
|
+
: 'Context : no generated context found — run sigmap first');
|
|
621
|
+
return out.join('\n');
|
|
622
|
+
}
|
|
623
|
+
|
|
589
624
|
/**
|
|
590
625
|
* get_callee_signatures — return the exact defining signature(s) of named
|
|
591
626
|
* symbols from the index, so an agent never guesses a callee's parameter types.
|
|
@@ -979,4 +1014,4 @@ function squeezeOutput(args, cwd) {
|
|
|
979
1014
|
return header + sq.squeezed;
|
|
980
1015
|
}
|
|
981
1016
|
|
|
982
|
-
module.exports = { readContext, searchSignatures, getMap, createCheckpoint, getRouting, explainFile, listModules, queryContext, getMethodImpact, getImpact, getLines, readMemory, getCalleeSignatures, notifyFileCreated, notifySymbolAdded, notifyFileDeleted, getDiffContext, getArchitectureOverview, verifySuggestion, squeezeOutput };
|
|
1017
|
+
module.exports = { readContext, searchSignatures, getMap, createCheckpoint, getRouting, explainFile, listModules, queryContext, getMethodImpact, getImpact, getLines, readMemory, getCalleeSignatures, notifyFileCreated, notifySymbolAdded, notifyFileDeleted, getDiffContext, getArchitectureOverview, verifySuggestion, squeezeOutput, getBudget };
|
package/src/mcp/server.js
CHANGED
|
@@ -14,11 +14,11 @@
|
|
|
14
14
|
|
|
15
15
|
const readline = require('readline');
|
|
16
16
|
const { TOOLS } = require('./tools');
|
|
17
|
-
const { readContext, searchSignatures, getMap, createCheckpoint, getRouting, explainFile, listModules, queryContext, getMethodImpact, getImpact, getLines, readMemory, getCalleeSignatures, notifyFileCreated, notifySymbolAdded, notifyFileDeleted, getDiffContext, getArchitectureOverview, verifySuggestion, squeezeOutput } = require('./handlers');
|
|
17
|
+
const { readContext, searchSignatures, getMap, createCheckpoint, getRouting, explainFile, listModules, queryContext, getMethodImpact, getImpact, getLines, readMemory, getCalleeSignatures, notifyFileCreated, notifySymbolAdded, notifyFileDeleted, getDiffContext, getArchitectureOverview, verifySuggestion, squeezeOutput, getBudget } = require('./handlers');
|
|
18
18
|
|
|
19
19
|
const SERVER_INFO = {
|
|
20
20
|
name: 'sigmap',
|
|
21
|
-
version: '8.
|
|
21
|
+
version: '8.23.0',
|
|
22
22
|
description: 'SigMap MCP server — code signatures on demand',
|
|
23
23
|
};
|
|
24
24
|
|
|
@@ -86,6 +86,7 @@ function dispatch(msg, cwd) {
|
|
|
86
86
|
else if (name === 'get_architecture_overview') text = getArchitectureOverview(args, cwd);
|
|
87
87
|
else if (name === 'verify_suggestion') text = verifySuggestion(args, cwd);
|
|
88
88
|
else if (name === 'squeeze_output') text = squeezeOutput(args, cwd);
|
|
89
|
+
else if (name === 'get_budget') text = getBudget(args, cwd);
|
|
89
90
|
else {
|
|
90
91
|
respondError(id, -32601, `Unknown tool: ${name}`);
|
|
91
92
|
return;
|
package/src/mcp/tools.js
CHANGED
|
@@ -387,6 +387,32 @@ const TOOLS = [
|
|
|
387
387
|
required: ['content'],
|
|
388
388
|
},
|
|
389
389
|
},
|
|
390
|
+
{
|
|
391
|
+
name: 'get_budget',
|
|
392
|
+
description:
|
|
393
|
+
'Session spend ledger — estimated tokens SigMap has emitted this session ' +
|
|
394
|
+
'(chars/4 estimates from the local gain log), optional budget remaining, and ' +
|
|
395
|
+
'context freshness. Scope honesty: counts only SigMap output, NOT the whole ' +
|
|
396
|
+
"chat's spend. Check this mid-session and degrade gracefully when near budget: " +
|
|
397
|
+
'prefer terse encoding, squeeze large outputs, summarize-then-drop context. ' +
|
|
398
|
+
'Local-only; no LLM, no network.',
|
|
399
|
+
inputSchema: {
|
|
400
|
+
type: 'object',
|
|
401
|
+
properties: {
|
|
402
|
+
session: {
|
|
403
|
+
type: 'string',
|
|
404
|
+
description:
|
|
405
|
+
'Session key to report on (default: SIGMAP_SESSION env or the UTC day bucket).',
|
|
406
|
+
},
|
|
407
|
+
budgetTokens: {
|
|
408
|
+
type: 'number',
|
|
409
|
+
description:
|
|
410
|
+
'Budget override in estimated tokens (default: config sessionBudgetTokens).',
|
|
411
|
+
},
|
|
412
|
+
},
|
|
413
|
+
required: [],
|
|
414
|
+
},
|
|
415
|
+
},
|
|
390
416
|
];
|
|
391
417
|
|
|
392
418
|
module.exports = { TOOLS };
|
|
@@ -0,0 +1,113 @@
|
|
|
1
|
+
'use strict';
|
|
2
|
+
|
|
3
|
+
/**
|
|
4
|
+
* Session spend ledger (v8.23 F1) — a queryable view over the existing gain
|
|
5
|
+
* log (.context/gain.ndjson, written by recordUsage).
|
|
6
|
+
*
|
|
7
|
+
* Honesty scope: this ledger counts tokens **SigMap emitted** (chars/4
|
|
8
|
+
* estimates), not the host chat's total spend — conversation history, model
|
|
9
|
+
* output, and other tools are invisible to a CLI. Every number here is an
|
|
10
|
+
* estimate and is labeled as such in output.
|
|
11
|
+
*
|
|
12
|
+
* Session identity: `SIGMAP_SESSION` env var when the host sets one, else a
|
|
13
|
+
* UTC day bucket (YYYY-MM-DD). Legacy gain entries (written before the
|
|
14
|
+
* `session` field existed) match day-bucket sessions by timestamp prefix.
|
|
15
|
+
*
|
|
16
|
+
* Zero dependencies; local JSON only.
|
|
17
|
+
*/
|
|
18
|
+
|
|
19
|
+
const fs = require('fs');
|
|
20
|
+
const path = require('path');
|
|
21
|
+
const { readGainLog } = require('./logger');
|
|
22
|
+
|
|
23
|
+
// Same generated-context surfaces cache/freshen.js watches.
|
|
24
|
+
const CONTEXT_PATHS = [
|
|
25
|
+
['.github', 'copilot-instructions.md'],
|
|
26
|
+
['CLAUDE.md'], ['AGENTS.md'], ['.github', 'context-cold.md'],
|
|
27
|
+
];
|
|
28
|
+
|
|
29
|
+
/** Session key: SIGMAP_SESSION override, else UTC day bucket. */
|
|
30
|
+
function sessionKey(env) {
|
|
31
|
+
const e = env || process.env;
|
|
32
|
+
if (e.SIGMAP_SESSION) return String(e.SIGMAP_SESSION);
|
|
33
|
+
return new Date().toISOString().slice(0, 10);
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
/** Newest mtime (ms) among generated context files, or 0 if none exist. */
|
|
37
|
+
function contextMtime(cwd) {
|
|
38
|
+
let newest = 0;
|
|
39
|
+
for (const parts of CONTEXT_PATHS) {
|
|
40
|
+
try { newest = Math.max(newest, fs.statSync(path.join(cwd, ...parts)).mtimeMs); } catch (_) {}
|
|
41
|
+
}
|
|
42
|
+
return newest;
|
|
43
|
+
}
|
|
44
|
+
|
|
45
|
+
/** Does a gain-log entry belong to this session? */
|
|
46
|
+
function entryInSession(entry, session) {
|
|
47
|
+
if (entry.session) return entry.session === session;
|
|
48
|
+
// Legacy entry: match day-bucket sessions on the timestamp date.
|
|
49
|
+
return /^\d{4}-\d{2}-\d{2}$/.test(session) && String(entry.ts || '').startsWith(session);
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
/**
|
|
53
|
+
* Session spend status.
|
|
54
|
+
* @param {string} cwd
|
|
55
|
+
* @param {object} [opts]
|
|
56
|
+
* @param {string} [opts.session] session key (default: sessionKey())
|
|
57
|
+
* @param {number} [opts.budgetTokens] budget override (else config sessionBudgetTokens)
|
|
58
|
+
* @param {number} [opts.contextTtlDays] TTL override (else config contextTtlDays)
|
|
59
|
+
* @param {object} [opts.config] loaded config (for the two keys above)
|
|
60
|
+
* @param {number} [opts.now] clock override for tests (ms)
|
|
61
|
+
* @returns {{
|
|
62
|
+
* session: string, unit: 'estimated-tokens', ops: number,
|
|
63
|
+
* spentTokens: number, baselineTokens: number, savedTokens: number,
|
|
64
|
+
* budgetTokens: number|null, remainingTokens: number|null, pctUsed: number|null,
|
|
65
|
+
* overBudget: boolean,
|
|
66
|
+
* context: { exists: boolean, ageMs: number|null, ageDays: number|null,
|
|
67
|
+
* ttlDays: number|null, stale: boolean }
|
|
68
|
+
* }}
|
|
69
|
+
*/
|
|
70
|
+
function budgetStatus(cwd, opts = {}) {
|
|
71
|
+
const session = opts.session || sessionKey();
|
|
72
|
+
const cfg = opts.config || {};
|
|
73
|
+
const budget = opts.budgetTokens != null ? Number(opts.budgetTokens)
|
|
74
|
+
: (Number.isFinite(cfg.sessionBudgetTokens) ? cfg.sessionBudgetTokens : null);
|
|
75
|
+
const ttlDays = opts.contextTtlDays != null ? Number(opts.contextTtlDays)
|
|
76
|
+
: (Number.isFinite(cfg.contextTtlDays) ? cfg.contextTtlDays : null);
|
|
77
|
+
const now = opts.now != null ? opts.now : Date.now();
|
|
78
|
+
|
|
79
|
+
let ops = 0, spent = 0, baseline = 0, saved = 0;
|
|
80
|
+
for (const e of readGainLog(cwd)) {
|
|
81
|
+
if (!entryInSession(e, session)) continue;
|
|
82
|
+
ops++;
|
|
83
|
+
spent += Number(e.actualTokens) || 0;
|
|
84
|
+
baseline += Number(e.baselineTokens) || 0;
|
|
85
|
+
saved += Number(e.savedTokens) || 0;
|
|
86
|
+
}
|
|
87
|
+
|
|
88
|
+
const mtime = contextMtime(cwd);
|
|
89
|
+
const ageMs = mtime > 0 ? Math.max(0, now - mtime) : null;
|
|
90
|
+
const ageDays = ageMs != null ? ageMs / 86400000 : null;
|
|
91
|
+
|
|
92
|
+
return {
|
|
93
|
+
session,
|
|
94
|
+
unit: 'estimated-tokens',
|
|
95
|
+
ops,
|
|
96
|
+
spentTokens: spent,
|
|
97
|
+
baselineTokens: baseline,
|
|
98
|
+
savedTokens: saved,
|
|
99
|
+
budgetTokens: budget,
|
|
100
|
+
remainingTokens: budget != null ? Math.max(0, budget - spent) : null,
|
|
101
|
+
pctUsed: budget > 0 ? Math.round((spent / budget) * 1000) / 10 : null,
|
|
102
|
+
overBudget: budget != null && spent > budget,
|
|
103
|
+
context: {
|
|
104
|
+
exists: mtime > 0,
|
|
105
|
+
ageMs,
|
|
106
|
+
ageDays: ageDays != null ? Math.round(ageDays * 10) / 10 : null,
|
|
107
|
+
ttlDays,
|
|
108
|
+
stale: ttlDays != null && ageDays != null && ageDays > ttlDays,
|
|
109
|
+
},
|
|
110
|
+
};
|
|
111
|
+
}
|
|
112
|
+
|
|
113
|
+
module.exports = { sessionKey, budgetStatus, contextMtime, entryInSession };
|
package/src/tracking/logger.js
CHANGED
|
@@ -175,10 +175,12 @@ function recordUsage(entry, cwd) {
|
|
|
175
175
|
const baseline = Math.max(0, Number(entry.baselineTokens) || 0);
|
|
176
176
|
const actual = Math.max(0, Number(entry.actualTokens) || 0);
|
|
177
177
|
const saved = Math.max(0, baseline - actual);
|
|
178
|
+
const ts = new Date().toISOString();
|
|
178
179
|
const record = {
|
|
179
|
-
ts
|
|
180
|
+
ts,
|
|
180
181
|
v: entry.version || '0.9.0',
|
|
181
182
|
op: entry.op || 'generate',
|
|
183
|
+
session: entry.session || process.env.SIGMAP_SESSION || ts.slice(0, 10),
|
|
182
184
|
baselineTokens: baseline,
|
|
183
185
|
actualTokens: actual,
|
|
184
186
|
savedTokens: saved,
|