tewip-ai 0.76.0 → 0.78.0

package/README.md

@@ -1216,6 +1216,39 @@ With `--act` it carries the decision out, and this is where the refusals matter
 - Everything it did or refused goes to `.tewip/actions.jsonl`, with the run that justified it.
 - It **will not repeat itself**: the same attempt is refused until a new run gives it new evidence.
 
+## Can everybody use the phone app? (Phase 143)
+
+```bash
+tewip access            # web markup and mobile screens, in one command
+```
+
+Every accessibility tool worth having points at a rendered page. A phone app has no page to point
+at without a simulator, a device farm or a build, so in practice nobody checks the mobile half -
+and the mobile half is where the two barriers that matter most are invisible by design. A fixed
+text size and a missing label both look perfect on the reviewer's screen. They appear when somebody
+turns their font size up, or turns VoiceOver on, which is after shipping.
+
+`tewip access` now reads **SwiftUI** and **Jetpack Compose** source alongside web markup:
+
+- **An image with no name.** SwiftUI: no `.accessibilityLabel` and nothing marking it decoration.
+  Compose: `contentDescription = null` inside something tappable, or a description that names
+  nothing - `"image"`, `"icon"`, `""`. An SF Symbol is left to a person, because VoiceOver reads
+  the symbol name and sometimes that is good enough.
+- **Text that will not grow.** `fontSize = 14.dp` in Compose ignores the font scale entirely; `sp`
+  honours it. `.font(.system(size: 13))` in SwiftUI opts out of Dynamic Type unless it carries
+  `relativeTo:`.
+- **A tap target smaller than a finger**, and only where something is actually tappable.
+- **Something made clickable with no role**, so TalkBack never says it is a control.
+
+**Measured on two real apps.** Google's Compose samples: **0 barriers**, 5 for a person to settle.
+isowords, a shipped SwiftUI game: 20 barriers, 62 to settle. Both found this check wrong first:
+
+- isowords has `Image(systemName: "xmark.circle.fill").font(.system(size: 24))`, where `.font` sets
+  the size of a glyph. Ten findings claimed text would not grow about something that is not text.
+- Google's JetNews has a `Row` with `.semantics { customActions = … }` - the correct handling, and
+  the reason the sample exists. A pattern wanting `semantics(` walked past the trailing-lambda form
+  everybody writes, and accused the reference implementation of the barrier it demonstrates fixing.
+
 ## What is your AI agent allowed to do? (Phase 139)
 
 ```bash
@@ -1252,6 +1285,14 @@ offline, in CI, before anything runs.
   because that is the difference between a wrong answer and a tool call somebody else chose. On
   1,691 files of real agent code: **zero findings**, which is the precision claim; recall is
   measured against planted examples and said to be.
+- **Which of these tools anybody wrote a test about.** Not "does this project have tests" - every
+  project has tests - but whether *this* tool is mentioned in any of them, and especially the ones
+  that cannot be undone. A name too short or too common to look for is reported as undecidable
+  rather than counted as covered. And the statement is made precisely in both directions: a tool
+  missing from the list is not a tool whose behaviour is untested, it is a tool nothing tests **by
+  that name** - the wiring between the name the model sees and the code behind it. Tewip finds 8 of
+  its own 17 that way. Also names which eval framework the project uses, and which it looked for
+  when it finds none.
 - **What the coding agent installed here was permitted to do.** `.claude/settings.json` and
   `.mcp.json` are committed, anybody with write access can change them, and neither turns up in a
   review the way a source file does. Tewip names the approval gate switched off

package/dist/engine/access/access-command.js

@@ -14,6 +14,7 @@ import { dirname, join, relative, resolve } from 'node:path';
 import { scanSourceFiles } from '../walk.js';
 import { readHandWritten } from '../../reporters/store.js';
 import { JS_LIKE, MARKUP, tags } from './markup.js';
+import { MOBILE, mobileBarriers } from './mobile.js';
 import { checkFile } from './checks.js';
 import { readAxeReport } from './reports.js';
 import { isLighthouseFile, readLighthouse } from '../perf/lighthouse.js';
@@ -48,7 +49,8 @@ export function findBarriers(root, only) {
     // scanner's 5,000-file cap over *all* text, which on n8n let 9 of 1,456 markup files through -
     // and none survived the test-file filter, so `access` reported no barriers having opened
     // nothing. A clean accessibility report is the one that must never be an artifact of a budget.
-    const scan = existsSync(base) && MARKUP.test(base) ? { files: [base], total: 1 } : scanSourceFiles(base, 20_000, (f) => MARKUP.test(f));
+    const reads = (f) => MARKUP.test(f) || MOBILE.test(f);
+    const scan = existsSync(base) && reads(base) ? { files: [base], total: 1 } : scanSourceFiles(base, 20_000, reads);
     const all = scan.files;
     // Asking for a path by name overrides the exclusion: if you point at a fixture, you want it read.
     const files = only ? all : all.filter((f) => !TEST_MARKUP.test(relative(root, f)));
@@ -66,6 +68,13 @@ export function findBarriers(root, only) {
         if (source.length > 400_000)
             continue;
         filesRead++;
+        const where = relative(root, path).replaceAll('\\', '/');
+        // A phone screen is markup too, and the same question - can everybody use this - but it is not
+        // written in tags, so it goes through its own reader rather than the tag extractor.
+        if (MOBILE.test(path)) {
+            findings.push(...mobileBarriers(where, source));
+            continue;
+        }
         const jsLike = JS_LIKE.test(path);
         // One page, rather than a file of components: only then do repeated ids land in one document.
         const isDocument = /\.html?$/i.test(path);

package/dist/engine/access/mobile.js

@@ -0,0 +1,212 @@
+const NL = String.fromCharCode(10);
+export const MOBILE = /\.(?:swift|kt)$/i;
+/** Apple's minimum is 44pt, Google's is 48dp. A control smaller than that is hard to hit. */
+const MIN_TAP = { swift: 44, kotlin: 48 };
+/** Placeholder text that is a label in form only. */
+const PLACEHOLDER = /^(?:|\s*|icon|image|picture|photo|img|button|todo|tbd|placeholder|null|none|n\/a|\.\.\.)$/i;
+const lineOf = (text, index) => text.slice(0, index).split(NL).length;
+const shown = (s) => s.replace(/\s+/g, ' ').trim().slice(0, 90);
+/**
+ * The call's own arguments, up to the matching bracket.
+ *
+ * Needed because a modifier that supplies the label sits *after* the call it labels -
+ * `Image("x").accessibilityLabel("A cat")` - so a check on the call alone would accuse every
+ * labelled image in the file.
+ */
+function callAndModifiers(text, from) {
+    let depth = 0;
+    let i = text.indexOf('(', from);
+    if (i === -1)
+        return text.slice(from, from + 120);
+    for (; i < text.length && i < from + 2000; i += 1) {
+        const ch = text[i];
+        if (ch === '(' || ch === '[' || ch === '{')
+            depth += 1;
+        else if (ch === ')' || ch === ']' || ch === '}') {
+            depth -= 1;
+            if (depth === 0)
+                break;
+        }
+    }
+    // Then the chained modifiers that follow, which is where SwiftUI puts accessibility.
+    let end = i + 1;
+    while (end < text.length && end < from + 4000) {
+        const rest = text.slice(end);
+        const chained = /^\s*\.\s*\w+\s*\(/.exec(rest);
+        if (!chained)
+            break;
+        let d = 0;
+        let j = end + rest.indexOf('(');
+        for (; j < text.length; j += 1) {
+            if (text[j] === '(')
+                d += 1;
+            else if (text[j] === ')') {
+                d -= 1;
+                if (d === 0)
+                    break;
+            }
+        }
+        end = j + 1;
+    }
+    return text.slice(from, end);
+}
+/**
+ * What a chained modifier is attached to.
+ *
+ * SwiftUI puts each modifier on its own line under the thing it modifies, so the owner is the
+ * first line above that does not itself begin with a dot. Needed because `.font(.system(size:))`
+ * means two different things depending on what it hangs off: a text size on a `Text`, and a glyph
+ * size on an `Image`.
+ */
+function chainOwner(text, at) {
+    const before = text.slice(0, at).split(NL);
+    for (let i = before.length - 1; i >= 0 && i > before.length - 8; i -= 1) {
+        const line = before[i].trim();
+        if (line === '' || line.startsWith('.'))
+            continue;
+        return line;
+    }
+    return '';
+}
+function swiftUI(file, text) {
+    const out = [];
+    // An image with no label and nothing saying it is decoration.
+    for (const m of text.matchAll(/\bImage\s*\(/g)) {
+        const at = m.index ?? 0;
+        const whole = callAndModifiers(text, at);
+        // `Image(decorative:)` is the documented way to say this conveys nothing. So is hiding it.
+        if (/\bdecorative\s*:/.test(whole) || /accessibilityHidden\s*\(\s*true\s*\)/.test(whole))
+            continue;
+        if (/accessibilityLabel\s*\(/.test(whole) || /accessibilityElement\s*\(/.test(whole))
+            continue;
+        // An SF Symbol carries a name VoiceOver can read; an asset name is a file name.
+        const symbol = /\bImage\s*\(\s*systemName\s*:/.test(whole);
+        out.push({
+            rule: symbol ? 'mobile-image-label' : 'mobile-image-label',
+            confidence: symbol ? 'ask-a-person' : 'barrier',
+            file,
+            line: lineOf(text, at),
+            element: shown(whole),
+            why: symbol
+                ? 'an SF Symbol with no accessibilityLabel: VoiceOver falls back to the symbol name, which is a developer name, not a description'
+                : 'no accessibilityLabel, and nothing marks it as decoration',
+            next: symbol
+                ? 'add .accessibilityLabel("…") saying what it conveys here, or .accessibilityHidden(true) if the text beside it already says it'
+                : 'add .accessibilityLabel("…"), or use Image(decorative:) if it conveys nothing',
+        });
+    }
+    // A text size that ignores the reader's font setting.
+    for (const m of text.matchAll(/\.\s*font\s*\(\s*\.\s*system\s*\(\s*size\s*:\s*(\d+)/g)) {
+        const at = m.index ?? 0;
+        const whole = callAndModifiers(text, at);
+        // `relativeTo:` opts the fixed size back into Dynamic Type, which is the documented fix.
+        if (/relativeTo\s*:/.test(whole))
+            continue;
+        // On an Image, `.font(.system(size:))` sets the size of an SF Symbol, not of any text. isowords
+        // has `Image(systemName: "xmark.circle.fill").font(.system(size: 24))`, and calling that text
+        // that will not grow is a confident wrong answer about a glyph.
+        if (/\bImage\s*\(/.test(chainOwner(text, at)))
+            continue;
+        out.push({
+            rule: 'mobile-text-not-scalable',
+            confidence: 'barrier',
+            file,
+            line: lineOf(text, at),
+            element: shown(m[0]),
+            why: `a fixed ${m[1]}pt size, so this text does not grow when the reader turns their font size up`,
+            next: 'use a semantic font like .font(.body), or .font(.system(size: 14, relativeTo: .body))',
+        });
+    }
+    // A tap target declared smaller than a finger.
+    for (const m of text.matchAll(/\.\s*frame\s*\(\s*width\s*:\s*(\d+)\s*,\s*height\s*:\s*(\d+)/g)) {
+        const w = Number(m[1]);
+        const h = Number(m[2]);
+        if (w >= MIN_TAP.swift && h >= MIN_TAP.swift)
+            continue;
+        const at = m.index ?? 0;
+        // Only when it is actually tappable: a small decorative frame is nobody's problem.
+        const around = text.slice(Math.max(0, at - 400), at + 400);
+        if (!/\bButton\s*\(|onTapGesture|\.\s*tapGesture|NavigationLink|\bToggle\s*\(/.test(around))
+            continue;
+        out.push({
+            rule: 'mobile-tap-target',
+            confidence: 'ask-a-person',
+            file,
+            line: lineOf(text, at),
+            element: shown(m[0]),
+            why: `a tappable area declared ${w}x${h}pt, under the 44pt Apple asks for - unless padding around it makes the hit area bigger, which this cannot see`,
+            next: 'give it .frame(minWidth: 44, minHeight: 44), or enough padding to reach 44pt',
+        });
+    }
+    return out;
+}
+function compose(file, text) {
+    const out = [];
+    // Compose makes contentDescription compulsory, so the failure is what gets passed to it.
+    for (const m of text.matchAll(/\b(?:Image|Icon)\s*\([^)]{0,400}?contentDescription\s*=\s*(null|"([^"]*)")/g)) {
+        const at = m.index ?? 0;
+        const whole = m[0];
+        const isNull = m[1] === 'null';
+        const value = m[2] ?? '';
+        // `null` is the documented way to say an image is decorative, and inside a labelled button it
+        // is right. On its own it is the commonest way an icon ends up with no name at all.
+        const around = text.slice(Math.max(0, at - 300), at + 300);
+        const insideClickable = /\b(?:IconButton|Button|Clickable|clickable)\b/.test(around);
+        if (isNull && !insideClickable)
+            continue;
+        if (!isNull && !PLACEHOLDER.test(value))
+            continue;
+        out.push({
+            rule: 'mobile-image-label',
+            confidence: isNull ? 'barrier' : 'barrier',
+            file,
+            line: lineOf(text, at),
+            element: shown(whole),
+            why: isNull
+                ? 'contentDescription = null inside something tappable: TalkBack announces a button with no name'
+                : `contentDescription is ${value === '' ? 'empty' : `"${value}"`}, which names nothing`,
+            next: 'give contentDescription the words a person needs, or keep null only when a text label beside it already says it',
+        });
+    }
+    // dp for a text size is always wrong: dp ignores the font scale, sp honours it.
+    for (const m of text.matchAll(/\bfontSize\s*=\s*(\d+)\s*\.\s*dp\b/g)) {
+        const at = m.index ?? 0;
+        out.push({
+            rule: 'mobile-text-not-scalable',
+            confidence: 'barrier',
+            file,
+            line: lineOf(text, at),
+            element: shown(m[0]),
+            why: `a text size in dp, which ignores the reader's font-size setting entirely. sp is the unit that honours it`,
+            next: `use ${m[1]}.sp instead of ${m[1]}.dp`,
+        });
+    }
+    // Something made tappable without being told it is a control.
+    // `clickable\s*[({]` and not `clickable\s*\(`: Kotlin's trailing lambda means the commonest form
+    // by far is `Modifier.clickable { open() }`, with a brace. Written with the paren alone this
+    // matched the form almost nobody uses and stayed silent on the one everybody does.
+    for (const m of text.matchAll(/\b(Box|Row|Column|Text|Surface|Spacer)\s*\(\s*[^)]{0,300}?Modifier\s*[^)]{0,300}?\.\s*clickable\s*[({]/g)) {
+        const at = m.index ?? 0;
+        const whole = callAndModifiers(text, at);
+        // `semantics\s*[({]` for the same reason as `clickable` above, and this one was caught by
+        // accusing Google's own JetNews sample of the exact thing it exists to demonstrate: its Row has
+        // `.semantics { customActions = listOf(CustomAccessibilityAction(...)) }`, with a brace, and a
+        // pattern wanting a paren walked straight past it. A check that reports the reference
+        // implementation as a barrier is not a strict check, it is a broken one.
+        if (/\bsemantics\s*[({]|\brole\s*=\s*Role\.|\bRole\.\w+|\bonClickLabel\s*=/.test(whole))
+            continue;
+        out.push({
+            rule: 'mobile-control-unnamed',
+            confidence: 'ask-a-person',
+            file,
+            line: lineOf(text, at),
+            element: shown(m[0]),
+            why: `a ${m[1]} made clickable with no role and no semantics, so TalkBack does not announce it as a control`,
+            next: 'pass Modifier.clickable(onClickLabel = "…", role = Role.Button), or use a Button',
+        });
+    }
+    return out;
+}
+export function mobileBarriers(file, text) {
+    return /\.swift$/i.test(file) ? swiftUI(file, text) : compose(file, text);
+}

package/dist/engine/access/wcag.js

@@ -7,6 +7,31 @@
 // trace to a numbered criterion is a finding they can put in front of a lawyer or a regulator; a
 // finding that only says "bad contrast" is not. So every rule carries its criterion and its level.
 export const RULES = {
+    // Phase 143: the mobile screens, which nobody checks because there is no page to point at.
+    'mobile-image-label': {
+        criterion: '1.1.1 Non-text Content',
+        level: 'A',
+        severity: 'blocker',
+        stops: 'VoiceOver or TalkBack announces "button" and stops, or reads out an asset file name',
+    },
+    'mobile-text-not-scalable': {
+        criterion: '1.4.4 Resize Text',
+        level: 'AA',
+        severity: 'blocker',
+        stops: 'somebody who turns their font size up gets the same small text, because the app opted out',
+    },
+    'mobile-tap-target': {
+        criterion: '2.5.8 Target Size (Minimum)',
+        level: 'AA',
+        severity: 'serious',
+        stops: 'anybody whose hands are unsteady, on a moving bus, or using one thumb',
+    },
+    'mobile-control-unnamed': {
+        criterion: '4.1.2 Name, Role, Value',
+        level: 'A',
+        severity: 'serious',
+        stops: 'a screen reader never says this is something you can press',
+    },
     'image-alt': {
         criterion: '1.1.1 Non-text Content',
         level: 'A',

package/dist/engine/agent/agent-command.js

@@ -1,56 +1,13 @@
 // `tewip agent`: what this agent is allowed to do, and what stands between it and the real world.
-import { readFileSync, writeFileSync } from 'node:fs';
-import { relative } from 'node:path';
-import { scanSourceFiles } from '../walk.js';
-import { CAPABILITY_MEANS, FRAMEWORKS_READ, IRREVERSIBLE, toolsIn, trifectaIn } from './tools.js';
-import { promptFlows } from './prompt-flow.js';
-import { configSection, scanConfig } from './claude-config.js';
-import { budgetSetToNothing, guardsIn, GUARD_MEANS, GUARD_PREVENTS } from './guards.js';
+import { writeFileSync } from 'node:fs';
+import { CAPABILITY_MEANS, FRAMEWORKS_READ, IRREVERSIBLE, trifectaIn } from './tools.js';
+import {} from './prompt-flow.js';
+import { configSection } from './claude-config.js';
+import { coverageSection } from './tool-tests.js';
+import { GUARD_MEANS, GUARD_PREVENTS } from './guards.js';
+import { scanAgent } from './scan.js';
+export { scanAgent };
 const NL = String.fromCharCode(10);
-const AGENT_FILE = /\.(?:py|[cm]?[jt]sx?)$/i;
-/**
- * What is not the shipped agent.
- *
- * The first version of this named the directory `tests` and the file `*.test.ts`, which is what the
- * documentation of every project says and not what projects do. Run against the OpenAI Agents SDK,
- * all 24 "tools" it reported were fixtures in `integration_tests/` and `test_*.py` - a directory
- * whose name ends in `tests` rather than being `tests`, and a Python convention that puts the word
- * at the front of the file. A list of an agent's tools that is actually a list of test doubles is
- * worse than no list.
- *
- * `[\w-]+[_-]tests?` and not `[\w-]*tests?`, because the second one also excludes `latest/`.
- */
-const NOT_SHIPPED = /(?:^|[\\/])(?:tests?|[\w-]+[_-]tests?|__tests__|spec|e2e|examples?|samples?|docs?|fixtures?|node_modules|\.venv|venv)[\\/]|\.(?:test|spec)\.[cm]?[jt]sx?$|_test\.py$|(?:^|[\\/])test_[^\\/]*\.py$/i;
-export function scanAgent(root) {
-    const scan = scanSourceFiles(root, 20_000, (f) => AGENT_FILE.test(f));
-    const tools = [];
-    const injections = [];
-    const guards = [];
-    let emptyBudget = null;
-    let filesRead = 0;
-    for (const path of scan.files) {
-        const where = relative(root, path).replace(/\\/g, '/');
-        if (NOT_SHIPPED.test(where))
-            continue;
-        let text;
-        try {
-            text = readFileSync(path, 'utf8');
-        }
-        catch {
-            continue;
-        }
-        filesRead += 1;
-        tools.push(...toolsIn(where, text));
-        injections.push(...promptFlows(where, text));
-        guards.push(...guardsIn(where, text));
-        if (!emptyBudget) {
-            const empty = budgetSetToNothing(text);
-            if (empty)
-                emptyBudget = { file: where, text: empty };
-        }
-    }
-    return { tools, injections, guards, filesRead, emptyBudget, config: scanConfig(root) };
-}
 const REACH_READS = {
     destructive: 'cannot be undone',
     deletes: 'deletes something',
@@ -62,7 +19,7 @@ const REACH_READS = {
     'not-sure': '**not decided**',
 };
 export function renderAgent(scan, root) {
-    const { tools, injections, guards, filesRead, emptyBudget, config } = scan;
+    const { tools, injections, guards, filesRead, emptyBudget, config, coverage, evals } = scan;
     const lines = ['# What this agent is allowed to do', ''];
     if (tools.length === 0) {
         return [
@@ -146,6 +103,7 @@ export function renderAgent(scan, root) {
         lines.push(`- **${capability}** (${CAPABILITY_MEANS[capability]}): ${legTools.length === 0 ? 'nothing found' : legTools.slice(0, 6).map((t) => `\`${t.name}\``).join(', ') + (legTools.length > 6 ? `, and ${legTools.length - 6} more` : '')}`);
     }
     lines.push('');
+    lines.push(...coverageSection(coverage, tools, evals));
     lines.push(...configSection(config));
     lines.push('## What stands between them and the real world', '');
     for (const kind of ['asks-a-person', 'dry-run', 'step-budget', 'tool-allow-list']) {

package/dist/engine/agent/scan.js

@@ -0,0 +1,84 @@
+// Reading an agent project once: its tools, its flows, its guards, its tests and its permissions.
+//
+// Split from agent-command.ts, which is the file in this project that keeps arriving at the
+// 300-line limit. The rendering belongs with the thing it renders; the walk belongs here.
+import { readFileSync } from 'node:fs';
+import { join, relative } from 'node:path';
+import { scanSourceFiles } from '../walk.js';
+import { toolsIn } from './tools.js';
+import { promptFlows } from './prompt-flow.js';
+import { budgetSetToNothing, guardsIn } from './guards.js';
+import { scanConfig } from './claude-config.js';
+import { coverageOf, evalEvidence } from './tool-tests.js';
+const NL = String.fromCharCode(10);
+const AGENT_FILE = /\.(?:py|[cm]?[jt]sx?)$/i;
+/**
+ * What is not the shipped agent.
+ *
+ * The first version of this named the directory `tests` and the file `*.test.ts`, which is what the
+ * documentation of every project says and not what projects do. Run against the OpenAI Agents SDK,
+ * all 24 "tools" it reported were fixtures in `integration_tests/` and `test_*.py` - a directory
+ * whose name ends in `tests` rather than being `tests`, and a Python convention that puts the word
+ * at the front of the file. A list of an agent's tools that is actually a list of test doubles is
+ * worse than no list.
+ *
+ * `[\w-]+[_-]tests?` and not `[\w-]*tests?`, because the second one also excludes `latest/`.
+ */
+const NOT_SHIPPED = /(?:^|[\\/])(?:tests?|[\w-]+[_-]tests?|__tests__|spec|e2e|examples?|samples?|docs?|fixtures?|node_modules|\.venv|venv)[\\/]|\.(?:test|spec)\.[cm]?[jt]sx?$|_test\.py$|(?:^|[\\/])test_[^\\/]*\.py$/i;
+export function scanAgent(root) {
+    const scan = scanSourceFiles(root, 20_000, (f) => AGENT_FILE.test(f));
+    const tools = [];
+    const tests = [];
+    const injections = [];
+    const guards = [];
+    let emptyBudget = null;
+    let filesRead = 0;
+    for (const path of scan.files) {
+        const where = relative(root, path).replace(/\\/g, '/');
+        // Excluded from the tool scan, but not thrown away: these files are the evidence for whether
+        // anybody ever wrote a test about a tool, which is a different question from what ships.
+        if (NOT_SHIPPED.test(where)) {
+            try {
+                tests.push({ file: where, text: readFileSync(path, 'utf8') });
+            }
+            catch {
+                // A file that cannot be read is not evidence of a test, and is not counted as one.
+            }
+            continue;
+        }
+        let text;
+        try {
+            text = readFileSync(path, 'utf8');
+        }
+        catch {
+            continue;
+        }
+        filesRead += 1;
+        tools.push(...toolsIn(where, text));
+        injections.push(...promptFlows(where, text));
+        guards.push(...guardsIn(where, text));
+        if (!emptyBudget) {
+            const empty = budgetSetToNothing(text);
+            if (empty)
+                emptyBudget = { file: where, text: empty };
+        }
+    }
+    const manifests = ['package.json', 'pyproject.toml', 'requirements.txt'].flatMap((m) => {
+        try {
+            return [readFileSync(join(root, m), 'utf8')];
+        }
+        catch {
+            return [];
+        }
+    });
+    return {
+        tools,
+        injections,
+        guards,
+        filesRead,
+        emptyBudget,
+        coverage: coverageOf(tools.map((t) => t.name), tests),
+        evals: evalEvidence(tests, manifests),
+        config: scanConfig(root),
+    };
+}

package/dist/engine/agent/tool-tests.js

@@ -0,0 +1,87 @@
+/**
+ * Eval frameworks, by the name they are imported or depended on under.
+ *
+ * Named rather than guessed at. The point of listing them is to be able to say "this project has
+ * one, here it is" or "none of these is here" - and the second sentence is only honest if the list
+ * is written down where a reader can see what was looked for.
+ */
+const FRAMEWORKS = [
+    { name: 'DeepEval', re: /\bdeepeval\b/i },
+    { name: 'promptfoo', re: /\bpromptfoo\b/i },
+    { name: 'Ragas', re: /\bragas\b/i },
+    { name: 'LangSmith', re: /\blangsmith\b/i },
+    { name: 'Braintrust', re: /\bbraintrust\b/i },
+    { name: 'Inspect AI', re: /\binspect_ai\b/i },
+    { name: 'Giskard', re: /\bgiskard\b/i },
+    { name: 'TruLens', re: /\btrulens\b/i },
+    { name: 'Langfuse', re: /\blangfuse\b/i },
+    { name: 'OpenAI Evals', re: /\bopenai[-_]?evals\b/i },
+];
+/** A path that holds an eval set rather than a unit test. */
+const EVAL_PATH = /(?:^|[\\/])(?:evals?|benchmarks?|golden|datasets?)[\\/]|\.eval\.[cm]?[jt]s$|_eval\.py$|eval_[^\\/]*\.(?:py|[cm]?[jt]s)$|\bevals?\.ya?ml$/i;
+/**
+ * Names too common to decide on.
+ *
+ * A tool called `get` or `run` appears in every file in the project, so finding it in a test file
+ * says nothing at all. Rather than counting those as covered - which is the comfortable direction
+ * and the wrong one - they are reported as undecidable, by name.
+ */
+const COMMON = new Set(['get', 'set', 'run', 'add', 'list', 'read', 'write', 'call', 'send', 'find', 'query', 'search', 'name', 'tool', 'test', 'echo', 'exec', 'open', 'close', 'load', 'save']);
+const tooCommonName = (name) => name.length < 5 || COMMON.has(name.toLowerCase());
+export function coverageOf(toolNames, tests) {
+    return toolNames.map((name) => {
+        if (tooCommonName(name))
+            return { name, mentionedIn: [], tooCommon: true };
+        // Word boundaries on both sides: `delete_user` must not be matched by `delete_user_profile`,
+        // which is a different tool and may well be the only one anybody tested.
+        const mention = new RegExp(String.raw `(?<![\w-])${name.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}(?![\w-])`);
+        return { name, mentionedIn: tests.filter((t) => mention.test(t.text)).map((t) => t.file), tooCommon: false };
+    });
+}
+export function evalEvidence(files, manifests) {
+    const frameworks = new Set();
+    for (const { name, re } of FRAMEWORKS) {
+        if (manifests.some((m) => re.test(m)) || files.some((f) => re.test(f.text)))
+            frameworks.add(name);
+    }
+    const evalFiles = files.filter((f) => EVAL_PATH.test(f.file)).map((f) => f.file);
+    return { frameworks: [...frameworks], files: [...new Set(evalFiles)].slice(0, 12) };
+}
+/** The names of every eval framework looked for, so silence about them can be read correctly. */
+export const FRAMEWORKS_LOOKED_FOR = FRAMEWORKS.map((f) => f.name);
+/**
+ * Which tools anybody wrote a test about, rendered where the logic for deciding it lives.
+ *
+ * Split out for the same reason the config section was: agent-command.ts is the file that keeps
+ * reaching the 300-line limit, and a report section belongs beside the thing it reports on.
+ */
+export function coverageSection(coverage, tools, evals) {
+    const lines = [];
+    const IRREV = ['destructive', 'deletes', 'pays', 'sends', 'runs-code'];
+    const untested = coverage.filter((c) => !c.tooCommon && c.mentionedIn.length === 0);
+    const riskyUntested = untested.filter((u) => IRREV.includes(tools.find((t) => t.name === u.name)?.reach ?? 'reads'));
+    lines.push('## Which of these anybody wrote a test about', '');
+    lines.push(untested.length === 0
+        ? 'Every tool whose name is distinctive enough to look for appears in at least one test file.'
+        : `**${untested.length} of ${tools.length}** appear in no test file${riskyUntested.length > 0 ? `, and ${riskyUntested.length} of those cannot be undone` : ''}.`, '');
+    if (untested.length > 0) {
+        for (const u of untested.slice(0, 15)) {
+            const reach = tools.find((t) => t.name === u.name)?.reach;
+            lines.push(`- \`${u.name}\`${reach && IRREV.includes(reach) ? ' — **and it cannot be undone**' : ''}`);
+        }
+        if (untested.length > 15)
+            lines.push(`…and ${untested.length - 15} more`);
+        lines.push('');
+    }
+    const undecidable = coverage.filter((c) => c.tooCommon);
+    if (undecidable.length > 0) {
+        lines.push(`Not decided for ${undecidable.length}: ${undecidable.slice(0, 8).map((u) => `\`${u.name}\``).join(', ')}. A name this`, 'short or this common appears everywhere, so finding it in a test file would prove nothing.', '');
+    }
+    lines.push(evals.frameworks.length > 0
+        ? `Eval framework in this project: ${evals.frameworks.join(', ')}.`
+        : `No eval framework found. Looked for: ${FRAMEWORKS_LOOKED_FOR.join(', ')}.`, '');
+    if (evals.files.length > 0)
+        lines.push(`Eval sets: ${evals.files.map((f) => `\`${f}\``).join(', ')}.`, '');
+    lines.push('A name appearing in a test file means somebody wrote it down there. Whether that test exercises', 'the tool, mocks it, or only imports it is not decided here.', '', 'And the other way round, which matters more: a tool missing from this list is not a tool whose', 'behaviour is untested. It is a tool nothing tests **by that name** - the wiring between the name', 'the model sees and the code behind it. Tewip finds 8 of its own 17 this way, and the engine under', 'each one is covered; what nothing exercises is the tool surface itself.', '');
+    return lines;
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "tewip-ai",
-  "version": "0.76.0",
+  "version": "0.78.0",
   "mcpName": "io.github.hmamut39/tewip",
   "description": "Tewip finds why each failing test failed - real bug, flaky test, environment, test data or changed selector - for any stack, from the command line, CI, or your editor.",
   "keywords": [