turntax 0.3.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -6,7 +6,7 @@ Coding agents re-send their instruction files with every turn. `turntax` measure
6
6
  npx turntax
7
7
  ```
8
8
 
9
- Try it in the browser, with nothing uploaded: https://torutdev.github.io/turntax/
9
+ Try it in the browser, with nothing uploaded: https://torutdev.github.io/turntax/ — drop a folder or enter a public GitHub repository. Links like `https://torutdev.github.io/turntax/?repo=owner/name` open straight to that repository's report.
10
10
 
11
11
  ## Supported agents
12
12
 
@@ -83,6 +83,23 @@ All checks are deterministic. Each one shows an estimated saving in o200k tokens
83
83
  - **stale-reference**: referenced files or `npm run` scripts that no longer exist.
84
84
  - **non-english**: non-English sections. The token multiplier is measured only against an English version you supply (`--english CLAUDE.md=CLAUDE.en.md`) or, if you opt in, a Claude translation (`--translate`).
85
85
 
86
+ ## Fixing
87
+
88
+ ```bash
89
+ npx turntax --fix # print a diff of the safe fixes; nothing is changed
90
+ npx turntax --fix --write # apply them
91
+ ```
92
+
93
+ `--fix` handles three kinds of advice:
94
+
95
+ - **duplicate**: removes lines from a file when every agent that reads the file also reads the same lines every turn from another file. Skill, command and subagent files are left whole.
96
+ - **scopable-section**: moves a section into the folder it is about: the same file name there (`packages/api/CLAUDE.md`), a nested `AGENTS.md`, or a Claude Code rule with `paths:`. The folder the section is about must already exist, and sections with `@imports` or relative links stay put. A section moves together with its subsections.
97
+ - **misconfigured**: renames `.md` files in `.cursor/rules` to `.mdc`.
98
+
99
+ Before a fix is kept, the files are classified again with the change applied: every agent must still read every line it read before, including files reached through `@imports`, and as often as before. The only exception is a moved section, which may load only when the agent works in that folder. Otherwise the fix is skipped and the reason is printed. A moved section can reach agents that did not read it before (a nested `AGENTS.md` is read by more agents than a `CLAUDE.md`); the summary names them. By default that check covers every supported agent, so a section of `AGENTS.md` usually stays where it is, because some agents read only the root file. `--tools codex,claude-code` limits the check to the agents you use.
100
+
101
+ In preview the diff is on stdout and works with `git apply`, and the summary is on stderr; with `--write` the summary is on stdout. `--write` checks that no file changed since it was read and writes nothing if one did. Symlinked files, their targets and files in symlinked folders are never changed. Contradictions and stale references need a person to decide, so `--fix` leaves them alone.
102
+
86
103
  ## MCP servers
87
104
 
88
105
  Every MCP server an agent starts adds its tool definitions (names, descriptions, input schemas) to the context. `turntax` reads the agents' MCP configs and lists the servers each agent starts:
@@ -148,7 +165,7 @@ jobs:
148
165
  runs-on: ubuntu-latest
149
166
  steps:
150
167
  - uses: actions/checkout@v4
151
- - uses: TorutDev/turntax@v0.3.0
168
+ - uses: TorutDev/turntax@v0.4.0
152
169
  with:
153
170
  max-increase: 500 # optional: fail when an agent grows by more than 500 tokens per turn
154
171
  # max-tokens: 8000 # optional: absolute budget
@@ -180,18 +197,22 @@ turntax [path]
180
197
  --translate opt-in Claude translation for that measurement (needs --claude-api)
181
198
  --prices <file.json> your own price table
182
199
  --section-tokens <n> threshold for oversized sections (default 600)
200
+ --fix print a diff of the safe fixes (--tools limits the agents whose context is kept)
201
+ --write with --fix, apply the changes
183
202
  ```
184
203
 
185
204
  ## Privacy
186
205
 
187
206
  Offline by default: no network calls, no telemetry. File contents are sent only when you pass `--claude-api`, `--gemini-api` or `--translate`, and only to that vendor's API. Keys are read from environment variables and sent only in request headers.
188
207
 
189
- `--mcp` runs the commands in the MCP configs it finds and connects to the configured HTTP servers, as the agents themselves would. Run it only on repositories whose configs you trust. `turntax` prints each command before starting it. `--include-user` reads files in your home directory and never sends them anywhere.
208
+ `--mcp` runs the commands in the MCP configs it finds and connects to the configured HTTP servers, as the agents themselves would. Run it only on repositories whose configs you trust. `turntax` prints each command before starting it. `--include-user` reads files in your home directory and never sends them anywhere. `--fix --write` is the only option that changes files, and only repository files.
190
209
 
191
210
  ## Web page
192
211
 
193
212
  [`web/`](web) is a static page that runs the same engine in the browser with the offline tokenizer only. Drop a repository folder or paste files; nothing is uploaded.
194
213
 
214
+ It can also load a public GitHub repository: it lists the files with the GitHub API (`api.github.com`, unauthenticated, 60 requests per hour) and downloads only the instruction files, and files they `@import`, from `raw.githubusercontent.com`. Those are the only network requests the page makes, and only when you load a repository or open a `?repo=` link.
215
+
195
216
  ```bash
196
217
  npm run build:web
197
218
  ```
package/dist/cli.js CHANGED
@@ -6,12 +6,14 @@ import { hasNonEnglish } from './core/advice.js';
6
6
  import { audit } from './core/audit.js';
7
7
  import { classify } from './core/classify.js';
8
8
  import { asDiffInput, diffReports, maxIncrease } from './core/diff.js';
9
+ import { planFixes, unifiedDiff } from './core/fix.js';
9
10
  import { parseMcpConfigs, serverKey, toolDefinitionsText, toolNamesText } from './core/mcp.js';
10
11
  import { DEFAULT_MODEL_IDS, MODELS, validateModels } from './core/prices.js';
11
12
  import { badgeMarkdown, headlineTokens, renderMarkdown, renderPrComment, renderText } from './core/render.js';
12
13
  import { countO200k } from './core/tokenize.js';
13
14
  import { TOOL_IDS } from './core/tools.js';
14
- import { discover, discoverMcpConfigs, discoverUser, existsIn, packageScripts } from './node/discover.js';
15
+ import { applyChanges } from './node/apply.js';
16
+ import { discover, discoverMcpConfigs, discoverUser, existsIn, linkedPaths, packageScripts } from './node/discover.js';
15
17
  import { filesAtRef } from './node/git.js';
16
18
  import { measureServer } from './node/mcp.js';
17
19
  import { anthropicCounter, geminiCounter, translateToEnglish } from './node/remote.js';
@@ -51,6 +53,10 @@ Options:
51
53
  (needs --claude-api; sends those files to api.anthropic.com)
52
54
  --prices <file.json> Use your own price table (same shape as --list-models --json)
53
55
  --section-tokens <n> Flag always-loaded sections at or above n o200k tokens (default: 600)
56
+ --fix Print a diff of the safe fixes: duplicate lines, sections that can load only in
57
+ their folder, misnamed Cursor rules. Every agent keeps reading the moved text;
58
+ --tools limits the agents whose context is kept
59
+ --write With --fix, apply the changes to the files
54
60
  -h, --help Show help
55
61
  -v, --version Show version
56
62
 
@@ -101,6 +107,8 @@ export async function main(argv, env = process.env) {
101
107
  baseline: { type: 'string' },
102
108
  compare: { type: 'string' },
103
109
  'max-increase': { type: 'string' },
110
+ fix: { type: 'boolean' },
111
+ write: { type: 'boolean' },
104
112
  help: { type: 'boolean', short: 'h' },
105
113
  version: { type: 'boolean', short: 'v' },
106
114
  },
@@ -144,6 +152,16 @@ export async function main(argv, env = process.env) {
144
152
  if (!TOOL_IDS.includes(t))
145
153
  throw new UsageError(`Unknown tool "${t}". Use: ${TOOL_IDS.join(', ')}`);
146
154
  const root = resolve(positionals[0] ?? '.');
155
+ if (o.write && !o.fix)
156
+ throw new UsageError('--write needs --fix.');
157
+ if (o.fix) {
158
+ const other = ['json', 'md', 'pr-comment', 'badge', 'baseline', 'compare', 'max-tokens', 'max-increase', 'include-user', 'mcp'].filter((k) => o[k] !== undefined);
159
+ if (o.write && o.out)
160
+ other.push('out');
161
+ if (other.length)
162
+ throw new UsageError(`--fix${o.write ? ' --write' : ''} cannot be combined with --${other.join(', --')}.`);
163
+ return await fix(root, tools, Boolean(o.write), o.out);
164
+ }
147
165
  const files = await discover(root);
148
166
  if (o.baseline && o.compare)
149
167
  throw new UsageError('Use --baseline or --compare, not both.');
@@ -276,6 +294,34 @@ export async function main(argv, env = process.env) {
276
294
  return 1;
277
295
  }
278
296
  }
297
+ async function fix(root, tools, write, out) {
298
+ const files = await discover(root);
299
+ const plan = planFixes(files, { tools, exists: existsIn(root), readOnly: linkedPaths(root, files) });
300
+ const log = (line) => (write ? process.stdout : process.stderr).write(line + '\n');
301
+ for (const f of plan.fixes)
302
+ log(`fix: ${f.message}${f.savingTokens ? ` [saves ~${f.savingTokens} tokens/turn]` : ''}`);
303
+ for (const s of plan.skipped)
304
+ log(`skipped ${s.kind}: ${s.reason}`);
305
+ if (!plan.changes.length) {
306
+ // An empty patch, so an earlier patch in the same file is not applied again.
307
+ if (out && !write)
308
+ writeFileSync(out, '');
309
+ log('Nothing to fix automatically.');
310
+ return 0;
311
+ }
312
+ if (write) {
313
+ applyChanges(root, plan.changes);
314
+ log(`Changed ${plan.changes.length} file(s): ${plan.changes.map((c) => c.path).join(', ')}`);
315
+ return 0;
316
+ }
317
+ const diff = unifiedDiff(plan.changes);
318
+ if (out)
319
+ writeFileSync(out, diff);
320
+ else
321
+ process.stdout.write(diff);
322
+ log('Preview only. Run again with --write to apply, or save the diff and use git apply.');
323
+ return 0;
324
+ }
279
325
  function nonNegativeInt(name, value) {
280
326
  const n = Number(value);
281
327
  if (!Number.isInteger(n) || n < 0)
@@ -265,11 +265,8 @@ function sections(units, threshold) {
265
265
  const tools = toolsWith(units, u.path, 'always');
266
266
  for (const s of splitSections(u.text)) {
267
267
  const tokens = countO200k(s.text);
268
- const refs = pathRefs(s.text).filter((r) => !r.startsWith('.'));
269
- const dirs = refs.map((r) => r.split('/')[0]);
270
- const top = mostCommon(dirs);
271
- if (tokens >= 100 && top && top.count >= 2 && top.count / dirs.length >= 0.6) {
272
- const scope = commonDir(refs.filter((r) => r.startsWith(top.value + '/')));
268
+ const scope = sectionScope(s.text);
269
+ if (scope) {
273
270
  out.push({
274
271
  kind: 'scopable-section',
275
272
  severity: 'medium',
@@ -297,6 +294,17 @@ function sections(units, threshold) {
297
294
  }
298
295
  return out;
299
296
  }
297
+ /** The directory a section is mostly about (for example `packages/api`), or null when it is not scopable. */
298
+ export function sectionScope(text) {
299
+ if (countO200k(text) < 100)
300
+ return null;
301
+ const refs = pathRefs(text).filter((r) => !r.startsWith('.'));
302
+ const dirs = refs.map((r) => r.split('/')[0]);
303
+ const top = mostCommon(dirs);
304
+ if (!top || top.count < 2 || top.count / dirs.length < 0.6)
305
+ return null;
306
+ return commonDir(refs.filter((r) => r.startsWith(top.value + '/')));
307
+ }
300
308
  /** Longest directory prefix shared by all refs, e.g. `packages/api` for `packages/api/db.ts` and `packages/api/handlers/`. */
301
309
  function commonDir(refs) {
302
310
  const dirs = refs.map((r) => (r.endsWith('/') ? r.slice(0, -1) : r.slice(0, r.lastIndexOf('/'))).split('/'));
@@ -0,0 +1,431 @@
1
+ import { sectionScope } from './advice.js';
2
+ import { classify } from './classify.js';
3
+ import { isInstructionFile } from './files.js';
4
+ import { basename, findImports, linesWithFences, parseFrontmatter, splitSections } from './markdown.js';
5
+ import { countO200k } from './tokenize.js';
6
+ import { TOOLS } from './tools.js';
7
+ const RANK = { ignored: 0, manual: 1, 'on-demand': 2, always: 3 };
8
+ const toolName = (id) => TOOLS.find((t) => t.id === id)?.name ?? id;
9
+ const names = (ids) => [...ids].map(toolName).join(', ');
10
+ /** Text with LF line endings, plus the line ending to write back. */
11
+ function lf(text) {
12
+ return { text: text.replace(/\r\n/g, '\n'), eol: text.includes('\r\n') ? '\r\n' : '\n' };
13
+ }
14
+ function contentUnits(units) {
15
+ return units.filter((u) => u.part !== 'description' && u.mode !== 'ignored');
16
+ }
17
+ /**
18
+ * A line as compared by the fixes: only list markers and whitespace are ignored. Case, underscores,
19
+ * asterisks and backticks can carry meaning (`create_user`, globs), so lines that differ there differ.
20
+ */
21
+ function lineKey(line) {
22
+ return line.replace(/^\s*(?:[-*+]|\d+[.)])\s+/, '').replace(/\s+/g, ' ').trim();
23
+ }
24
+ /** Prose lines long enough to be meaningful (outside fenced code, not headings), by key. */
25
+ function proseLines(text) {
26
+ const out = new Map();
27
+ for (const { line, inFence } of linesWithFences(text)) {
28
+ const key = lineKey(line);
29
+ if (!inFence && key.length >= 30 && !key.startsWith('#'))
30
+ out.set(key, line);
31
+ }
32
+ return out;
33
+ }
34
+ function keysOf(text) {
35
+ return text
36
+ .split('\n')
37
+ .map(lineKey)
38
+ .filter((k) => k.replace(/[#\s]/g, '').length > 0);
39
+ }
40
+ function loadedLines(units) {
41
+ const out = new Map();
42
+ for (const u of contentUnits(units)) {
43
+ const lines = out.get(u.tool) ?? new Map();
44
+ out.set(u.tool, lines);
45
+ for (const key of keysOf(u.text))
46
+ lines.set(key, Math.max(lines.get(key) ?? 0, RANK[u.mode]));
47
+ }
48
+ return out;
49
+ }
50
+ /**
51
+ * Agents that would lose something: every line an agent read before must still be read at least as
52
+ * often, except the `edited` lines, which need only `need(before)`.
53
+ */
54
+ function check(before, after, edited, need) {
55
+ const lost = [];
56
+ for (const [tool, lines] of before) {
57
+ const now = after.get(tool);
58
+ for (const [key, rank] of lines) {
59
+ if ((now?.get(key) ?? 0) < (edited.has(key) ? need(rank) : rank)) {
60
+ lost.push(tool);
61
+ break;
62
+ }
63
+ }
64
+ }
65
+ return lost;
66
+ }
67
+ class Workspace {
68
+ tools;
69
+ state;
70
+ constructor(input, tools) {
71
+ this.tools = tools;
72
+ this.state = this.build(new Map(input.map((f) => [f.path, f])));
73
+ }
74
+ build(files) {
75
+ const units = classify([...files.values()], this.tools);
76
+ return { files, units, loaded: loadedLines(units) };
77
+ }
78
+ get files() {
79
+ return this.state.files;
80
+ }
81
+ get units() {
82
+ return this.state.units;
83
+ }
84
+ /** The state with `edits` applied (null deletes a file). */
85
+ trial(edits) {
86
+ const files = new Map(this.state.files);
87
+ for (const [path, text] of edits) {
88
+ if (text === null)
89
+ files.delete(path);
90
+ else
91
+ files.set(path, { path, text });
92
+ }
93
+ return this.build(files);
94
+ }
95
+ check(next, edited, need) {
96
+ return check(this.state.loaded, next.loaded, new Set(edited), need);
97
+ }
98
+ commit(next) {
99
+ this.state = next;
100
+ }
101
+ }
102
+ export function planFixes(input, opts = {}) {
103
+ // Only repository files are ever changed; user-level files (~/...) are not passed in.
104
+ const ws = new Workspace(input.filter((f) => !f.path.startsWith('~')), opts.tools);
105
+ // Files reached only through @imports (a README, docs) are for people too; they are never changed.
106
+ const readOnly = (path) => !isInstructionFile(path)
107
+ ? `${path} is not an agent instruction file (it is @imported); it is left alone.`
108
+ : opts.readOnly?.(path)
109
+ ? `${path} is a symlink, in a symlinked folder or the target of a symlink; it is left alone.`
110
+ : null;
111
+ const original = new Map(ws.files);
112
+ const renames = new Map();
113
+ const fixes = [];
114
+ const skipped = [];
115
+ if (!opts.tools || opts.tools.includes('cursor'))
116
+ renameCursorRules(ws, readOnly, renames, fixes, skipped);
117
+ // Moves run before duplicate removal: a move may relax how often lines load, so a line must not be
118
+ // removed elsewhere on the strength of a copy that is moved later.
119
+ moveScopableSections(ws, opts, readOnly, fixes, skipped);
120
+ removeDuplicates(ws, readOnly, fixes, skipped);
121
+ const renamedTo = new Map([...renames].map(([from, to]) => [to, from]));
122
+ const changes = [];
123
+ for (const [path, file] of ws.files) {
124
+ const from = renamedTo.get(path);
125
+ const before = original.get(from ?? path)?.text ?? null;
126
+ if (from)
127
+ changes.push({ path, before, after: file.text, renamedFrom: from });
128
+ else if (before !== file.text)
129
+ changes.push({ path, before, after: file.text });
130
+ }
131
+ changes.sort((a, b) => a.path.localeCompare(b.path));
132
+ return { fixes, skipped, changes };
133
+ }
134
+ /** `.md` files in `.cursor/rules` are ignored by Cursor; `.mdc` is the documented extension. */
135
+ function renameCursorRules(ws, readOnly, renames, fixes, skipped) {
136
+ for (const f of [...ws.files.values()]) {
137
+ if (!/^(?:.*\/)?\.cursor\/rules\/.+\.md$/.test(f.path))
138
+ continue;
139
+ const to = f.path.replace(/\.md$/, '.mdc');
140
+ if (readOnly(f.path) || ws.files.has(to)) {
141
+ skipped.push({ kind: 'misconfigured', paths: [f.path], reason: readOnly(f.path) ?? `${to} already exists.` });
142
+ continue;
143
+ }
144
+ const next = ws.trial(new Map([[f.path, null], [to, f.text]]));
145
+ const lost = ws.check(next, [], (r) => r);
146
+ if (lost.length) {
147
+ skipped.push({ kind: 'misconfigured', paths: [f.path], reason: `${names(lost)} would stop reading it.` });
148
+ continue;
149
+ }
150
+ const mode = next.units.find((u) => u.tool === 'cursor' && u.path === to && u.part !== 'description')?.mode;
151
+ ws.commit(next);
152
+ renames.set(f.path, to);
153
+ fixes.push({
154
+ kind: 'misconfigured',
155
+ paths: [f.path, to],
156
+ message: `Renamed ${f.path} to ${basename(to)} so Cursor reads it` +
157
+ (mode === 'manual' ? '; without frontmatter it applies only when @-mentioned (add alwaysApply, globs or description).' : '.'),
158
+ savingTokens: null,
159
+ });
160
+ }
161
+ }
162
+ /** Indexes of body lines (after any frontmatter) outside fenced code. */
163
+ function bodyLines(text) {
164
+ const start = text.split('\n').length - parseFrontmatter(text).body.split('\n').length;
165
+ return linesWithFences(text)
166
+ .map(({ line, inFence }, index) => ({ line, inFence, index }))
167
+ .filter((l) => !l.inFence && l.index >= start);
168
+ }
169
+ /** Drops the given line indexes; a blank line left next to another blank line where a line was removed goes too. */
170
+ function dropLines(lines, drop) {
171
+ const kept = [];
172
+ let removed = false;
173
+ lines.forEach((line, i) => {
174
+ if (drop.has(i)) {
175
+ removed = true;
176
+ return;
177
+ }
178
+ if (removed && line.trim() === '' && (kept.length === 0 || kept[kept.length - 1].trim() === ''))
179
+ return;
180
+ removed = false;
181
+ kept.push(line);
182
+ });
183
+ return kept;
184
+ }
185
+ /**
186
+ * Removes a line from a file when every agent that reads the file also reads the same line every turn
187
+ * from another file. Files read by fewer agents are trimmed first, so the broadest copy stays.
188
+ */
189
+ function removeDuplicates(ws, readOnly, fixes, skipped) {
190
+ const readers = (path) => new Set(contentUnits(ws.units).filter((u) => u.path === path).map((u) => u.tool));
191
+ const order = [...ws.files.keys()].sort((a, b) => readers(a).size - readers(b).size || b.localeCompare(a));
192
+ for (const path of order) {
193
+ const tools = readers(path);
194
+ // Skill, command and subagent bodies are procedures or separate prompts; they are left whole.
195
+ if (!tools.size || ws.units.some((u) => u.path === path && u.part === 'body'))
196
+ continue;
197
+ const { text, eol } = lf(ws.files.get(path).text);
198
+ const elsewhere = (key) => contentUnits(ws.units).filter((u) => u.path !== path && u.mode === 'always' && proseLines(u.text).has(key));
199
+ // Lines this file shares with another file that some agent reads every turn.
200
+ const shared = new Map();
201
+ const blockers = new Set();
202
+ const removable = new Set();
203
+ for (const [key] of proseLines(text)) {
204
+ const others = elsewhere(key);
205
+ if (!others.length)
206
+ continue;
207
+ shared.set(key, [...new Set(others.map((u) => u.path))]);
208
+ const missing = [...tools].filter((t) => !others.some((u) => u.tool === t));
209
+ missing.forEach((t) => blockers.add(t));
210
+ if (!missing.length)
211
+ removable.add(key);
212
+ }
213
+ if (!shared.size)
214
+ continue;
215
+ const otherPaths = [...new Set([...shared.values()].flat())];
216
+ if (readOnly(path)) {
217
+ skipped.push({ kind: 'duplicate', paths: [path, ...otherPaths], reason: readOnly(path) });
218
+ continue;
219
+ }
220
+ if (removable.size) {
221
+ const lines = text.split('\n');
222
+ const drop = new Set(bodyLines(text).filter((l) => removable.has(lineKey(l.line))).map((l) => l.index));
223
+ const removed = [...drop].map((i) => lines[i]);
224
+ const next = ws.trial(new Map([[path, dropLines(lines, drop).join('\n').replace(/\n/g, eol)]]));
225
+ const lost = ws.check(next, removable, (r) => r);
226
+ if (lost.length) {
227
+ skipped.push({ kind: 'duplicate', paths: [path, ...otherPaths], reason: `Kept ${removable.size} line(s) in ${path}: ${names(lost)} would lose context without them (for example a file they @import).` });
228
+ }
229
+ else {
230
+ const everyTurn = contentUnits(ws.units).some((u) => u.path === path && u.mode === 'always');
231
+ const from = [...new Set([...removable].flatMap((k) => shared.get(k)))];
232
+ ws.commit(next);
233
+ fixes.push({
234
+ kind: 'duplicate',
235
+ paths: [path, ...from],
236
+ message: `Removed ${removed.length} line(s) from ${path} that ${names(tools)} already read every turn from ${from.join(', ')}.`,
237
+ savingTokens: everyTurn ? countO200k(removed.join('\n')) : 0,
238
+ });
239
+ }
240
+ }
241
+ const kept = shared.size - removable.size;
242
+ if (kept && blockers.size)
243
+ skipped.push({
244
+ kind: 'duplicate',
245
+ paths: [path, ...otherPaths],
246
+ reason: `Kept ${kept} line(s) in ${path} that are also in ${otherPaths.join(', ')}: ${names(blockers)} would lose them.`,
247
+ });
248
+ }
249
+ }
250
+ /** Where a section about `scope` can live so it loads only there: the same file name nested, AGENTS.md, or a Claude rule. */
251
+ function targets(source, scope, heading) {
252
+ const name = basename(source);
253
+ const slug = heading.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, '') || 'section';
254
+ const out = [];
255
+ if (['CLAUDE.md', 'AGENTS.md', 'GEMINI.md'].includes(name))
256
+ out.push({ path: `${scope}/${name}`, header: '' });
257
+ if (name !== 'AGENTS.md')
258
+ out.push({ path: `${scope}/AGENTS.md`, header: '' });
259
+ out.push({ path: `.claude/rules/${slug}.md`, header: `---\npaths:\n - "${scope}/**"\n---\n\n` });
260
+ return out;
261
+ }
262
+ const headingLevel = (line) => /^\s{0,3}(#{1,6})\s/.exec(line)?.[1].length ?? 0;
263
+ /** Level 2 and 3 sections of the LF text, each running to the next heading of the same or a higher level. */
264
+ function movableSections(text) {
265
+ const lines = text.split('\n');
266
+ const all = splitSections(text).filter((s) => s.heading !== '(top)');
267
+ return all
268
+ .map((s) => ({ s, level: headingLevel(lines[s.line - 1]) }))
269
+ .filter((x) => x.level >= 2)
270
+ .map(({ s, level }) => {
271
+ const next = all.find((o) => o.line > s.line && headingLevel(lines[o.line - 1]) <= level);
272
+ const end = next ? next.line - 1 : lines.length;
273
+ return { heading: s.heading, start: s.line - 1, end, body: lines.slice(s.line - 1, end).join('\n').replace(/\n+$/, '') };
274
+ });
275
+ }
276
+ function moveScopableSections(ws, opts, readOnly, fixes, skipped) {
277
+ const sources = [...new Set(ws.units.filter((u) => u.mode === 'always' && u.part === 'full').map((u) => u.path))].sort();
278
+ const tried = new Set();
279
+ for (const path of sources) {
280
+ // Sections are found in the raw file so that comments and formatting move as they are.
281
+ for (;;) {
282
+ const { text, eol } = lf(ws.files.get(path).text);
283
+ const next = movableSections(text)
284
+ .map((x) => ({ ...x, scope: sectionScope(x.body) }))
285
+ .find((x) => x.scope && !tried.has(`${path}\0${x.body}`));
286
+ if (!next)
287
+ break;
288
+ tried.add(`${path}\0${next.body}`);
289
+ const { heading, body, start, end } = next;
290
+ const scope = next.scope;
291
+ const reason = readOnly(path) ?? moveBlocker(body, scope, opts);
292
+ if (reason) {
293
+ skipped.push({ kind: 'scopable-section', paths: [path], reason: `Section "${heading}": ${reason}` });
294
+ continue;
295
+ }
296
+ const lines = text.split('\n');
297
+ const head = lines.slice(0, start);
298
+ const tail = lines.slice(end);
299
+ while (head.length && tail.length && head[head.length - 1].trim() === '' && tail[0].trim() === '')
300
+ tail.shift();
301
+ const rest = [...head, ...tail].join('\n').replace(/\n*$/, '\n');
302
+ let moved = false;
303
+ const refused = [];
304
+ for (const t of targets(path, scope, heading)) {
305
+ const existing = ws.files.get(t.path);
306
+ if (t.path === path || readOnly(t.path) || (existing && t.header))
307
+ continue;
308
+ const target = existing ? lf(existing.text) : { text: '', eol };
309
+ const targetText = existing ? `${target.text.replace(/\n+$/, '')}\n\n${body}\n` : `${t.header}${body}\n`;
310
+ const trial = ws.trial(new Map([
311
+ [path, rest.replace(/\n/g, eol)],
312
+ [t.path, targetText.replace(/\n/g, target.eol)],
313
+ ]));
314
+ // Agents that loaded the section every turn may now load it only when working in that folder.
315
+ const lost = ws.check(trial, keysOf(body), (r) => Math.min(r, RANK['on-demand']));
316
+ if (lost.length) {
317
+ refused.push(`${t.path} (${names(lost)} would lose context)`);
318
+ continue;
319
+ }
320
+ const readers = [...new Set(ws.units.filter((u) => u.path === path && u.mode === 'always' && u.part !== 'description').map((u) => u.tool))];
321
+ const reads = (l, tool) => keysOf(body).some((k) => l.get(tool)?.has(k));
322
+ const added = [...trial.loaded.keys()].filter((tool) => !reads(ws.state.loaded, tool) && reads(trial.loaded, tool));
323
+ ws.commit(trial);
324
+ fixes.push({
325
+ kind: 'scopable-section',
326
+ paths: [path, t.path],
327
+ message: `Moved section "${heading}" from ${path} to ${t.path}${existing ? '' : ' (new file)'}; ${names(readers)} load it only when working in ${scope}/.` +
328
+ (added.length ? ` Now also read there by ${names(added)}.` : ''),
329
+ savingTokens: countO200k(body),
330
+ });
331
+ moved = true;
332
+ break;
333
+ }
334
+ if (!moved)
335
+ skipped.push({
336
+ kind: 'scopable-section',
337
+ paths: [path],
338
+ reason: `Section "${heading}": no target keeps every agent's context: ${refused.join('; ')}. Use --tools to choose the agents to keep it for.`,
339
+ });
340
+ }
341
+ }
342
+ }
343
+ function moveBlocker(body, scope, opts) {
344
+ if (scope.split('/').some((seg) => seg === '' || seg === '.' || seg === '..'))
345
+ return `unusual folder ${scope}.`;
346
+ if (opts.exists && !opts.exists(scope))
347
+ return `folder ${scope}/ does not exist.`;
348
+ if (findImports(body).length)
349
+ return 'it has @imports, which resolve relative to the file they are in.';
350
+ if (/\]\((?!https?:|#|\/|mailto:)[^)]+\)/.test(body))
351
+ return 'it has relative links, which resolve relative to the file they are in.';
352
+ return null;
353
+ }
354
+ /** Unified diff of the planned changes, in the format `git apply` accepts. */
355
+ export function unifiedDiff(changes) {
356
+ let out = '';
357
+ for (const c of changes) {
358
+ const from = c.renamedFrom ?? c.path;
359
+ out += `diff --git a/${from} b/${c.path}\n`;
360
+ if (c.renamedFrom) {
361
+ if (c.before === c.after)
362
+ out += 'similarity index 100%\n';
363
+ out += `rename from ${c.renamedFrom}\nrename to ${c.path}\n`;
364
+ if (c.before === c.after)
365
+ continue;
366
+ }
367
+ if (c.before === null)
368
+ out += 'new file mode 100644\n';
369
+ out += `--- ${c.before === null ? '/dev/null' : `a/${from}`}\n+++ b/${c.path}\n`;
370
+ out += hunks(c.before ?? '', c.after);
371
+ }
372
+ return out;
373
+ }
374
+ function split(text) {
375
+ if (text === '')
376
+ return [];
377
+ const lines = text.split('\n');
378
+ const noEol = lines[lines.length - 1] !== '';
379
+ if (!noEol)
380
+ lines.pop();
381
+ return lines.map((t, i) => ({ text: t, noEol: noEol && i === lines.length - 1 }));
382
+ }
383
+ /** Line diff by longest common subsequence; instruction files are small. */
384
+ function diffLines(a, b) {
385
+ const n = a.length;
386
+ const m = b.length;
387
+ const same = (i, j) => a[i].text === b[j].text && a[i].noEol === b[j].noEol;
388
+ if (n * m > 4_000_000)
389
+ return [...a.map((l) => ({ op: '-', ...l })), ...b.map((l) => ({ op: '+', ...l }))];
390
+ const lcs = Array.from({ length: n + 1 }, () => new Uint32Array(m + 1));
391
+ for (let i = n - 1; i >= 0; i--)
392
+ for (let j = m - 1; j >= 0; j--)
393
+ lcs[i][j] = same(i, j) ? lcs[i + 1][j + 1] + 1 : Math.max(lcs[i + 1][j], lcs[i][j + 1]);
394
+ const out = [];
395
+ let i = 0;
396
+ let j = 0;
397
+ while (i < n || j < m) {
398
+ if (i < n && j < m && same(i, j)) {
399
+ out.push({ op: ' ', ...a[i++] });
400
+ j++;
401
+ }
402
+ else if (i < n && (j === m || lcs[i + 1][j] >= lcs[i][j + 1]))
403
+ out.push({ op: '-', ...a[i++] });
404
+ else
405
+ out.push({ op: '+', ...b[j++] });
406
+ }
407
+ return out;
408
+ }
409
+ function hunks(before, after, context = 3) {
410
+ const lines = diffLines(split(before), split(after));
411
+ const changed = lines.map((l, i) => (l.op === ' ' ? -1 : i)).filter((i) => i >= 0);
412
+ let out = '';
413
+ let k = 0;
414
+ while (k < changed.length) {
415
+ const start = Math.max(0, changed[k] - context);
416
+ let end = Math.min(lines.length, changed[k] + context + 1);
417
+ while (k + 1 < changed.length && changed[k + 1] - context <= end)
418
+ end = Math.min(lines.length, changed[++k] + context + 1);
419
+ k++;
420
+ const slice = lines.slice(start, end);
421
+ const prefix = lines.slice(0, start);
422
+ const oldStart = prefix.filter((l) => l.op !== '+').length;
423
+ const newStart = prefix.filter((l) => l.op !== '-').length;
424
+ const oldLen = slice.filter((l) => l.op !== '+').length;
425
+ const newLen = slice.filter((l) => l.op !== '-').length;
426
+ out += `@@ -${oldLen ? oldStart + 1 : oldStart},${oldLen} +${newLen ? newStart + 1 : newStart},${newLen} @@\n`;
427
+ for (const l of slice)
428
+ out += `${l.op}${l.text}\n${l.noEol ? '\\n' : ''}`;
429
+ }
430
+ return out;
431
+ }
@@ -0,0 +1,43 @@
1
+ import { lstatSync, mkdirSync, readFileSync, realpathSync, renameSync, writeFileSync } from 'node:fs';
2
+ import { dirname, isAbsolute, relative, resolve } from 'node:path';
3
+ import { realLocation } from './discover.js';
4
+ const inside = (root, p) => {
5
+ const rel = relative(root, p);
6
+ return !rel.startsWith('..') && !isAbsolute(rel);
7
+ };
8
+ /**
9
+ * Writes planned changes under `root`. Every file is checked first: files to change must still have the
10
+ * content the plan was made from, new files must not exist, and nothing may be written through a symlink
11
+ * (dangling ones included) or outside the repository. Nothing is written when a check fails.
12
+ */
13
+ export function applyChanges(root, changes) {
14
+ const realRoot = realpathSync(root);
15
+ const exists = (p) => lstatSync(p, { throwIfNoEntry: false }) !== undefined;
16
+ const abs = (path) => {
17
+ const p = resolve(root, path);
18
+ if (lstatSync(p, { throwIfNoEntry: false })?.isSymbolicLink())
19
+ throw new Error(`${path} is a symlink; nothing was written.`);
20
+ const real = realLocation(p);
21
+ if (!inside(root, p) || real === null || !inside(realRoot, real))
22
+ throw new Error(`${path} is outside ${root}; nothing was written.`);
23
+ return p;
24
+ };
25
+ for (const c of changes) {
26
+ const source = c.renamedFrom ?? c.path;
27
+ if (c.renamedFrom && exists(abs(c.path)))
28
+ throw new Error(`${c.path} already exists; nothing was written.`);
29
+ if (c.before === null ? exists(abs(c.path)) : !exists(abs(source)) || readFileSync(abs(source), 'utf8') !== c.before)
30
+ throw new Error(`${source} ${c.before === null ? 'already exists' : 'changed since it was read'}; nothing was written.`);
31
+ }
32
+ for (const c of changes) {
33
+ const target = abs(c.path);
34
+ mkdirSync(dirname(target), { recursive: true });
35
+ if (c.renamedFrom)
36
+ renameSync(abs(c.renamedFrom), target);
37
+ // New files are created exclusively, so a link placed in the meantime is not followed.
38
+ if (c.before === null)
39
+ writeFileSync(target, c.after, { flag: 'wx' });
40
+ else if (c.after !== c.before || !c.renamedFrom)
41
+ writeFileSync(target, c.after);
42
+ }
43
+ }
@@ -1,5 +1,5 @@
1
- import { existsSync, lstatSync, readdirSync, readFileSync, statSync } from 'node:fs';
2
- import { join } from 'node:path';
1
+ import { existsSync, lstatSync, readdirSync, readFileSync, realpathSync, statSync } from 'node:fs';
2
+ import { dirname, isAbsolute, join, relative, resolve, sep } from 'node:path';
3
3
  import { MAX_DEPTH, addImports, isInstructionFile, isSkippedDir } from '../core/files.js';
4
4
  import { MCP_CONFIGS, USER_MCP_PATHS } from '../core/mcp.js';
5
5
  import { USER_ROOTS, isUserInstructionFile } from '../core/user.js';
@@ -106,3 +106,36 @@ export function discoverUser(home) {
106
106
  });
107
107
  return { files: [...files.values()].sort((a, b) => a.path.localeCompare(b.path)), mcpConfigs };
108
108
  }
109
+ /** The real location of `p`, following symlinks in the part of the path that exists; null behind a dangling symlink. */
110
+ export function realLocation(p) {
111
+ if (lstatSync(p, { throwIfNoEntry: false })) {
112
+ try {
113
+ return realpathSync(p);
114
+ }
115
+ catch {
116
+ return null;
117
+ }
118
+ }
119
+ const parent = dirname(p);
120
+ if (parent === p)
121
+ return p;
122
+ const real = realLocation(parent);
123
+ return real === null ? null : resolve(real, relative(parent, p));
124
+ }
125
+ /**
126
+ * A check for paths that must not be changed: paths that are symlinks or lie under a symlinked folder
127
+ * (existing or not), and repository files that such a link points to.
128
+ */
129
+ export function linkedPaths(root, files) {
130
+ const realRoot = realpathSync(root);
131
+ const linked = (path) => realLocation(join(root, path)) !== join(realRoot, path);
132
+ const targets = new Set();
133
+ for (const f of files) {
134
+ if (!linked(f.path))
135
+ continue;
136
+ const rel = relative(realRoot, realLocation(join(root, f.path)) ?? realRoot);
137
+ if (!rel.startsWith('..') && !isAbsolute(rel))
138
+ targets.add(rel.split(sep).join('/'));
139
+ }
140
+ return (path) => targets.has(path) || linked(path);
141
+ }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "turntax",
3
- "version": "0.3.0",
3
+ "version": "0.4.0",
4
4
  "description": "Measure the per-turn token cost of AI coding-agent instruction files (AGENTS.md, CLAUDE.md, Cursor, Copilot, Cline, Roo, Kiro, Windsurf and more) for every agent that reads them.",
5
5
  "license": "MIT",
6
6
  "author": "TORUT LTD",