tewip-ai 0.6.0 → 0.7.0

package/README.md

@@ -160,9 +160,31 @@ pixels, colours or spacing, because a design tool and a browser do not measure t
 No model is called, so it costs nothing and answers the same way every time.
 
 Measured on four real design/page pairs (a public repository's Figma exports and the pages built
-from them): 38 checks, **0 differences reported on pages that match their design**, and **8 of 8
-injected drifts caught** - a renamed button, a dropped sentence, a region that stopped being marked
-up. No designer has judged the output yet, so the roadmap's own metric is still open.
+from them): 38 checks, **0 differences reported on pages that match their design**, **11 of 11 real
+drifts caught** (a renamed control, a dropped sentence, a word swapped inside a sentence, a region
+that lost its markup) and **5 of 5 harmless changes ignored** (capitals, extra spaces, a button
+implemented as a link, reordered sections). No designer has judged the output yet, so the roadmap's
+own metric is still open.
+
+## Security: which findings actually matter (Phase 9, new)
+
+```bash
+tewip secure                 # secrets + npm audit / pip-audit / osv-scanner, triaged
+tewip secure --secrets-only  # a fast check before you commit
+tewip secure --gate          # fails a build only on what needs a person now
+```
+
+Scanners are good at finding things and bad at saying which of them matter. Tewip runs the ones your
+project already has and sorts every finding into **act on this now**, **real but not reachable**
+(a dev dependency, a package nothing imports, a file git does not track), **noise** (a placeholder
+from a README), or **accepted** (a person already judged it, in a file only people write). Each
+verdict carries the evidence. No model is called, so it costs nothing, and **a credential is
+located, never printed**.
+
+Measured on the gitleaks project's own labelled samples, for the credential kinds Tewip covers:
+**50 of 54 real credentials found**. Four defects that corpus exposed are fixed; the four still
+missed are things that are not credentials (an unsigned token, a Stripe *test* key). It never claims
+a project is secure - it prints what it checked and what it did not.
 
 ## Epics, stories and acceptance criteria (Phase 14, new)
 

package/TEAM-GUIDE.md

@@ -121,6 +121,7 @@ collects environment failures in one issue, and (optionally) opens a fix pull re
 | `tewip write` | Turns the plan's scenarios into real tests in your own framework, and keeps only the ones that pass repeatedly **and** fail when their check is broken on purpose | When you know what to test but not how to write it |
 | `tewip plan --issue PROJ-123` | Plans from the Jira ticket itself: its description and acceptance criteria | "Here's the story, what should we test?" |
 | `tewip jira plan.json --project PROJ` | Files the plan's scenarios as test-case tickets, linked to the story | Handing work to the team |
+| `tewip secure` | Runs your dependency and secret scanners, then says which findings need someone now and which are noise or unreachable | Before a release, or in CI |
 | `tewip spec "<feature>"` | An epic, its user stories and acceptance criteria, grounded in your code; files them in Jira with `--project PROJ --create` | Before the work is broken down |
 | `tewip report report.xml --gate` | The CI command for GitLab, Jenkins, CircleCI, Azure, Bitbucket and the rest: diagnoses the run, comments on the GitLab merge request, and fails the build only on failures that need a person | Your CI is not GitHub Actions |
 | `tewip watch --cmd "npm test"` | Runs your tests when you save a file and explains each failure straight away | While you are writing code |
@@ -136,6 +137,8 @@ Notes that save you time:
 - `write` never touches your application code. It only adds new test files, and you review and
   commit them yourself.
 - Everything it writes locally goes to the `.tewip` folder in your project.
+- Jira works with **Cloud, Server and Data Center**. To read tickets from a tracker anyone can read
+  (many open-source ones), `TEWIP_JIRA_URL` alone is enough - no account. Writing needs a token.
 - `tewip jira` shows what it *would* file and creates nothing until you add `--create`. It never
   files the same scenario twice, and never moves, assigns or closes a ticket.
 - `tewip report` posts to Slack or Teams only when a failure needs a person (`TEWIP_SLACK_WEBHOOK`,
@@ -186,7 +189,10 @@ Notes that save you time:
   design says and what the page says, side by side, so a renamed button or a dropped sentence is
   found before review.
 
-**Security** — planned, not built yet (`tewip secure`, Phase 9 in ROADMAP.md).
+**Security**
+- `tewip secure` triages what the scanners find: what is committed and reachable, what is only in a
+  dev dependency, what is a placeholder in a README. A credential is never printed, only located,
+  and `.tewip/security-accepted.txt` records what your team has already judged.
 
 Managers, architects and leads do not need to install anything: those views are produced by the CI
 job and kept in a GitHub issue, or can be asked for through an AI assistant.

package/dist/cli/tewip.js

@@ -38,6 +38,11 @@ Usage:
                                     File the plan's scenarios as test-case issues, linked to the
                                     story. Shows what it would create; --create actually files
                                     them, and the same scenario is never filed twice.
+  tewip secure [--secrets-only] [--gate] [--out security.md]
+                                    Runs the scanners this project already has (npm audit,
+                                    pip-audit, osv-scanner) plus Tewip's own secret scan, then says
+                                    which findings need someone now and which are noise or not
+                                    reachable. No model, no cost, and it never prints a credential.
   tewip spec "<feature>" [--diff main...HEAD] [--url <page> ...] [--project PROJ] [--create]
                                     An epic, its user stories and their acceptance criteria, from
                                     this project's code, tests and failure history. Anything it
@@ -151,6 +156,19 @@ async function main() {
             });
             return;
         }
+        case 'secure': {
+            const { runSecure } = await import('../engine/secure/secure-command.js');
+            const items = await runSecure({
+                root: process.cwd(),
+                out: flag(args, 'out'),
+                secretsOnly: args.includes('--secrets-only'),
+                gate: args.includes('--gate'),
+            });
+            // The gate fails a build only for what Tewip decided a person must act on now.
+            if (args.includes('--gate') && items.some((i) => i.verdict === 'real-risk'))
+                process.exitCode = 1;
+            return;
+        }
         case 'spec': {
             const feature = args.filter((a, i) => !a.startsWith('--') && !['--diff', '--url', '--out', '--project'].includes(args[i - 1] ?? '')).join(' ').trim();
             if (!feature)

package/dist/engine/secure/deps.js

@@ -0,0 +1,123 @@
+// Phase 9: known-vulnerable dependencies. Tewip runs no scanner of its own here - the ecosystems
+// already have good ones, and their databases are updated daily by people whose job that is. It
+// runs whichever is available (npm, pip, or OSV for everything else), reads their output, and then
+// does the part they do not: say which findings actually matter in this project.
+import { existsSync } from 'node:fs';
+import { join } from 'node:path';
+import { spawnSync } from 'node:child_process';
+const run = (command, cwd, timeoutMs = 5 * 60_000) => {
+    const r = spawnSync(command, { cwd, shell: true, encoding: 'utf8', timeout: timeoutMs, maxBuffer: 40 * 1024 * 1024 });
+    return { code: r.status ?? 1, out: `${r.stdout ?? ''}`, err: `${r.stderr ?? ''}` };
+};
+const has = (command, cwd) => run(`${command} --version`, cwd, 30_000).code === 0;
+const SEVERITY_ORDER = ['critical', 'high', 'moderate', 'medium', 'low', 'info'];
+export const bySeverity = (a, b) => SEVERITY_ORDER.indexOf(a.severity.toLowerCase()) - SEVERITY_ORDER.indexOf(b.severity.toLowerCase());
+/** `npm audit --json`, which needs no install and no account: every Node project already has it. */
+export function npmAudit(root) {
+    if (!existsSync(join(root, 'package.json')))
+        return [];
+    const { out } = run('npm audit --json', root);
+    let report;
+    try {
+        report = JSON.parse(out);
+    }
+    catch {
+        return [];
+    }
+    const findings = [];
+    for (const [name, entry] of Object.entries(report.vulnerabilities ?? {})) {
+        const via = (entry.via ?? []).find((v) => typeof v === 'object');
+        findings.push({
+            package: name,
+            version: String(entry.range ?? ''),
+            severity: String(entry.severity ?? 'unknown'),
+            id: String(via?.url?.split('/').pop() ?? via?.source ?? ''),
+            title: String(via?.title ?? `${name} has a known vulnerability`),
+            fixedIn: entry.fixAvailable && typeof entry.fixAvailable === 'object' ? String(entry.fixAvailable.version ?? '') : '',
+            source: 'npm audit',
+            dev: Boolean(entry.isDirect === false ? false : entry.dev ?? false) || Boolean(entry.effects?.length === 0 && entry.dev),
+        });
+    }
+    return findings;
+}
+/** `pip-audit`, when the project is Python and the tool is installed. */
+export function pipAudit(root) {
+    const python = ['requirements.txt', 'pyproject.toml', 'setup.py'].some((f) => existsSync(join(root, f)));
+    if (!python || !has('pip-audit', root))
+        return [];
+    const { out } = run('pip-audit -f json --progress-spinner off', root);
+    try {
+        const report = JSON.parse(out);
+        const list = report.dependencies ?? report;
+        return (Array.isArray(list) ? list : []).flatMap((dep) => (dep.vulns ?? []).map((v) => ({
+            package: String(dep.name ?? ''),
+            version: String(dep.version ?? ''),
+            severity: String(v.severity ?? 'unknown').toLowerCase(),
+            id: String(v.id ?? ''),
+            title: String(v.description ?? '').split('\n')[0].slice(0, 200),
+            fixedIn: (v.fix_versions ?? []).join(', '),
+            source: 'pip-audit',
+            dev: false,
+        })));
+    }
+    catch {
+        return [];
+    }
+}
+/** OSV-Scanner, which covers Go, Rust, Java, Ruby, PHP and more, when it is installed. */
+export function osvScan(root) {
+    if (!has('osv-scanner', root))
+        return [];
+    const { out } = run(`osv-scanner --format json --recursive "${root}"`, root);
+    try {
+        const report = JSON.parse(out);
+        const findings = [];
+        for (const result of report.results ?? []) {
+            for (const pkg of result.packages ?? []) {
+                for (const v of pkg.vulnerabilities ?? []) {
+                    findings.push({
+                        package: String(pkg.package?.name ?? ''),
+                        version: String(pkg.package?.version ?? ''),
+                        severity: String(v.database_specific?.severity ?? v.severity?.[0]?.type ?? 'unknown').toLowerCase(),
+                        id: String(v.id ?? ''),
+                        title: String(v.summary ?? '').slice(0, 200),
+                        fixedIn: '',
+                        source: 'osv-scanner',
+                        dev: false,
+                    });
+                }
+            }
+        }
+        return findings;
+    }
+    catch {
+        return [];
+    }
+}
+export function auditDependencies(root) {
+    const ran = [];
+    const missing = [];
+    const findings = [];
+    if (existsSync(join(root, 'package.json'))) {
+        ran.push('npm audit');
+        findings.push(...npmAudit(root));
+    }
+    const python = ['requirements.txt', 'pyproject.toml', 'setup.py'].some((f) => existsSync(join(root, f)));
+    if (python) {
+        if (has('pip-audit', root)) {
+            ran.push('pip-audit');
+            findings.push(...pipAudit(root));
+        }
+        else {
+            missing.push('pip-audit (pip install pip-audit) for the Python dependencies');
+        }
+    }
+    if (has('osv-scanner', root)) {
+        ran.push('osv-scanner');
+        findings.push(...osvScan(root));
+    }
+    else if (!existsSync(join(root, 'package.json')) && !python) {
+        missing.push('osv-scanner (https://osv.dev) for this project\'s ecosystem');
+    }
+    return { findings: findings.sort(bySeverity), ran, missing };
+}

package/dist/engine/secure/secrets.js

@@ -0,0 +1,158 @@
+// Phase 9: secrets left in a repository. This is the one security check Tewip does itself, because
+// it needs no tool installed and no network: a credential either matches a shape its issuer
+// publishes, or it does not.
+//
+// Two rules keep it honest. A finding must quote where it is, never what it is: the value is
+// redacted everywhere, including in the report and in anything posted to CI. And a shape alone is
+// not a risk - "AKIAIOSFODNN7EXAMPLE" is in Amazon's own documentation - so a match that looks like
+// a placeholder, or that has too little randomness to be a real key, is reported as noise rather
+// than as a leak.
+import { readdirSync, readFileSync, statSync } from 'node:fs';
+import { join, relative, sep } from 'node:path';
+/** Shapes their own issuers publish. Each one is a credential, not a guess about one. */
+const PATTERNS = [
+    { kind: 'aws-access-key-id', what: 'an AWS access key id', re: /\b((?:AKIA|ASIA|AGPA|AIDA|AROA|ANPA)[A-Z0-9]{16})\b/g, minEntropy: 2.5 },
+    { kind: 'aws-secret-key', what: 'an AWS secret access key', re: /\baws_secret_access_key\s*[=:]\s*['"]?([A-Za-z0-9/+=]{40})\b/gi },
+    { kind: 'github-token', what: 'a GitHub token', re: /\b((?:ghp|gho|ghu|ghs|ghr)_[A-Za-z0-9]{36,255}|github_pat_[A-Za-z0-9_]{60,})\b/g },
+    { kind: 'gitlab-token', what: 'a GitLab personal access token', re: /\b(glpat-[A-Za-z0-9_-]{20,})(?![A-Za-z0-9_-])/g },
+    { kind: 'slack-token', what: 'a Slack token', re: /\b(xox[baprs]-[A-Za-z0-9-]{10,})\b/g },
+    { kind: 'slack-webhook', what: 'a Slack incoming webhook', re: /(https:\/\/hooks\.slack\.com\/services\/[A-Za-z0-9/+]{20,})/g },
+    // Stripe writes both "live" and "prod" on real keys; a "test" key is not a credential, and is
+    // left alone on purpose. The prod form was missed until a labelled sample showed it.
+    { kind: 'stripe-key', what: 'a live Stripe key', re: /\b((?:sk|rk)_(?:live|prod)_[A-Za-z0-9]{20,})(?![A-Za-z0-9])/g },
+    // A key may end in "-", where a word boundary never matches: a real Google key was missed that
+    // way. And "sk-ant-..." is Anthropic's, not OpenAI's, so the looser shape must exclude it or one
+    // credential gets reported twice under two names.
+    { kind: 'openai-key', what: 'an OpenAI API key', re: /\b(sk-(?:proj-|svcacct-|admin-)?(?!ant-)[A-Za-z0-9_-]{32,})(?![A-Za-z0-9_-])/g },
+    { kind: 'anthropic-key', what: 'an Anthropic API key', re: /\b(sk-ant-(?:api|admin)[0-9]{2}-[A-Za-z0-9_-]{32,})(?![A-Za-z0-9_-])/g },
+    { kind: 'google-api-key', what: 'a Google API key', re: /\b(AIza[A-Za-z0-9_-]{35})(?![A-Za-z0-9_-])/g },
+    { kind: 'npm-token', what: 'an npm token', re: /\b(npm_[A-Za-z0-9]{36})\b/g },
+    { kind: 'pypi-token', what: 'a PyPI token', re: /\b(pypi-AgEIcHlwaS5vcmc[A-Za-z0-9_-]{50,})\b/g },
+    { kind: 'sendgrid-key', what: 'a SendGrid key', re: /\b(SG\.[A-Za-z0-9_-]{20,}\.[A-Za-z0-9_-]{20,})\b/g },
+    { kind: 'twilio-key', what: 'a Twilio key', re: /\b(SK[a-f0-9]{32})\b/g },
+    { kind: 'digitalocean-token', what: 'a DigitalOcean token', re: /\b(dop_v1_[a-f0-9]{64})\b/g },
+    { kind: 'private-key', what: 'a private key', re: /(-----BEGIN (?:RSA |EC |DSA |OPENSSH |PGP )?PRIVATE KEY(?: BLOCK)?-----)/g, minEntropy: 0 },
+    { kind: 'connection-string', what: 'a database URL with a password in it', re: /\b((?:postgres|postgresql|mysql|mongodb(?:\+srv)?|redis|amqp):\/\/[^:\s'"]+:([^@\s'"]{6,})@[^\s'"]+)/g },
+    { kind: 'jwt', what: 'a signed token (JWT)', re: /\b(eyJ[A-Za-z0-9_-]{10,}\.eyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,})(?![A-Za-z0-9_-])/g },
+    // A password in a web address leaks the same way a database URL does, and is easier to miss.
+    { kind: 'url-credentials', what: 'a password inside a URL', re: /\b(https?:\/\/[^:/\s'"]+:([^@\s'"]{4,})@[^\s'"]+)/g, minEntropy: 2 },
+];
+/** How random a value is, in bits per character. A real key is above 3; "xxxxxxxx" is 0. */
+export function entropyOf(value) {
+    if (!value)
+        return 0;
+    const counts = new Map();
+    for (const ch of value)
+        counts.set(ch, (counts.get(ch) ?? 0) + 1);
+    let bits = 0;
+    for (const n of counts.values()) {
+        const p = n / value.length;
+        bits -= p * Math.log2(p);
+    }
+    return bits;
+}
+const PLACEHOLDER_WORDS = /example|placeholder|dummy|sample|changeme|your[-_ ]?(?:key|token|secret|api)|xxxx|redacted|fake|notreal|test[-_]?key|<[^>]+>|\$\{|\{\{/i;
+/** Files whose whole purpose is to show the shape of a secret, not to hold one. */
+const DOC_FILE = /(?:^|[\\/])(?:\.env\.example|\.env\.sample|\.env\.template|[^\\/]*\.(?:md|rst|txt|adoc)|[^\\/]*example[^\\/]*|[^\\/]*sample[^\\/]*)$/i;
+/** Why this match is probably not a live credential, or null when nothing says so. */
+export function looksPlanted(value, line, file) {
+    if (PLACEHOLDER_WORDS.test(value))
+        return 'the value itself says it is an example';
+    if (PLACEHOLDER_WORDS.test(line))
+        return 'the line around it says it is an example';
+    if (DOC_FILE.test(file))
+        return 'it is in a documentation or example file';
+    if (/(.)\1{5,}/.test(value))
+        return 'the value repeats one character, so it is a stand-in';
+    return null;
+}
+const SKIP_DIR = new Set(['node_modules', '.git', 'dist', 'build', 'out', 'coverage', 'target', 'vendor', '__pycache__', '.venv', 'venv', '.tewip', '.next', '.cache']);
+const BINARY = /\.(?:png|jpe?g|gif|webp|ico|svg|pdf|zip|gz|tar|exe|dll|so|dylib|woff2?|ttf|eot|mp4|mp3|wav|class|jar|pyc|lock)$/i;
+const MAX_FILE_BYTES = 2 * 1024 * 1024;
+/** Every text file in a project, skipping what no repository owns. */
+export function sourceFiles(root, limit = 5000) {
+    const out = [];
+    const walk = (dir) => {
+        if (out.length >= limit)
+            return;
+        let entries;
+        try {
+            entries = readdirSync(dir);
+        }
+        catch {
+            return;
+        }
+        for (const entry of entries) {
+            if (out.length >= limit)
+                return;
+            const path = join(dir, entry);
+            let info;
+            try {
+                info = statSync(path);
+            }
+            catch {
+                continue;
+            }
+            if (info.isDirectory()) {
+                if (!SKIP_DIR.has(entry))
+                    walk(path);
+                continue;
+            }
+            if (BINARY.test(entry) || info.size > MAX_FILE_BYTES)
+                continue;
+            out.push(path);
+        }
+    };
+    walk(root);
+    return out;
+}
+/** The value, blanked: first two characters, then dots. Tewip never prints a credential. */
+export function redact(line, value) {
+    const shown = value.slice(0, 2);
+    const hidden = `${shown}${'.'.repeat(Math.min(12, Math.max(4, value.length - 2)))}`;
+    return line.replace(value, hidden).trim().slice(0, 200);
+}
+export function scanSecrets(root, files = sourceFiles(root)) {
+    const findings = [];
+    const seen = new Set();
+    for (const file of files) {
+        let text;
+        try {
+            text = readFileSync(file, 'utf8');
+        }
+        catch {
+            continue;
+        }
+        if (text.includes('\u0000'))
+            continue; // binary after all
+        const lines = text.split('\n');
+        for (const { kind, what, re, minEntropy } of PATTERNS) {
+            for (const match of text.matchAll(re)) {
+                const value = match[1];
+                const entropy = entropyOf(value);
+                if (entropy < (minEntropy ?? 3))
+                    continue;
+                const index = match.index ?? 0;
+                const line = text.slice(0, index).split('\n').length;
+                const rel = relative(root, file).split(sep).join('/');
+                // One credential, one finding: a value can match a loose shape and a tighter one, and the
+                // same key reported under two names reads as two leaks.
+                const key = `${rel}:${line}:${value}`;
+                if (seen.has(key) || seen.has(`${rel}:${line}`))
+                    continue;
+                seen.add(key);
+                seen.add(`${rel}:${line}`);
+                findings.push({
+                    kind,
+                    what,
+                    file: rel,
+                    line,
+                    evidence: redact(lines[line - 1] ?? '', value),
+                    entropy: Number(entropy.toFixed(2)),
+                    placeholder: looksPlanted(value, lines[line - 1] ?? '', rel),
+                });
+            }
+        }
+    }
+    return findings;
+}

package/dist/engine/secure/secure-command.js

@@ -0,0 +1,79 @@
+// `tewip secure`: run the scanners a project already has, then say which of their findings matter.
+//
+// It costs nothing (no model is called), it never prints a credential, and it never claims a
+// project is secure: it says exactly what was checked, what was not, and what it decided about each
+// finding. "No real risk found" is a statement about these checks, not a certificate.
+import { mkdirSync, writeFileSync } from 'node:fs';
+import { dirname, join, resolve } from 'node:path';
+import { auditDependencies } from './deps.js';
+import { scanSecrets, sourceFiles } from './secrets.js';
+import { readAccepted, triageDependency, triageSecret, worstFirst } from './triage.js';
+const CODE = /\.(?:[cm]?[jt]sx?|py|rb|go|java|kt|cs|php|rs|vue|svelte)$/;
+export function renderSecurity(items, checked, notChecked, root) {
+    const real = items.filter((i) => i.verdict === 'real-risk');
+    const later = items.filter((i) => i.verdict === 'not-reachable');
+    const noise = items.filter((i) => i.verdict === 'noise');
+    const accepted = items.filter((i) => i.verdict === 'accepted');
+    const lines = ['# Tewip security check', ''];
+    lines.push(`Checked: ${checked.length > 0 ? checked.join(', ') : 'nothing ran'}.`);
+    if (notChecked.length > 0)
+        lines.push(`**Not checked**: ${notChecked.join('; ')}.`);
+    lines.push('');
+    lines.push(`${real.length} need someone now · ${later.length} real but not reachable · ${noise.length} noise · ${accepted.length} accepted earlier.`);
+    lines.push('');
+    const block = (title, group, explain) => {
+        if (group.length === 0)
+            return;
+        lines.push(`## ${title} (${group.length})`, '', explain, '');
+        for (const item of group) {
+            lines.push(`- **${item.title}** · ${item.severity}`);
+            lines.push(`  - Why: ${item.why}`);
+            lines.push(`  - Do: ${item.next}`);
+            lines.push(`  - Found by: ${item.source}`);
+        }
+        lines.push('');
+    };
+    block('Act on these', real, 'Each one is committed or reachable, and the evidence is beside it.');
+    block('Real, but not reachable here', later, 'True of the repository, not of what it runs: fix them on a normal schedule, not tonight.');
+    block('Noise', noise, 'A shape without a credential behind it. Listed so you can see nothing was hidden.');
+    block('Accepted earlier', accepted, `From ${join('.tewip', 'security-accepted.txt')}, written by a person.`);
+    lines.push('---', '');
+    lines.push('Tewip never prints a credential: a secret is located, never quoted. To accept a finding,');
+    lines.push(`add a line to \`${join(root, '.tewip', 'security-accepted.txt')}\`, for example:`);
+    lines.push('```', 'secret tests/fixtures/sample.env:3 - a fixture, not a real key', 'dependency lodash - waiting for the upstream fix', '```');
+    lines.push('');
+    lines.push('This is not a certificate. It is the result of the checks named above, and nothing else.');
+    return lines.join('\n');
+}
+export async function runSecure(cmd) {
+    const root = resolve(cmd.root);
+    const accepted = readAccepted(root);
+    const files = sourceFiles(root);
+    const items = [];
+    const checked = ['secrets in the working tree'];
+    const notChecked = [];
+    for (const finding of scanSecrets(root, files))
+        items.push(triageSecret(finding, root, accepted));
+    if (!cmd.secretsOnly) {
+        const audit = auditDependencies(root);
+        checked.push(...audit.ran);
+        notChecked.push(...audit.missing);
+        const sources = files.filter((f) => CODE.test(f));
+        for (const finding of audit.findings)
+            items.push(triageDependency(finding, root, sources, accepted));
+    }
+    else {
+        notChecked.push('dependencies (--secrets-only was asked for)');
+    }
+    // A web scan is a different kind of check and needs a running app; it is not pretended here.
+    notChecked.push('a running application (no web scan is part of this command yet)');
+    items.sort(worstFirst);
+    const report = renderSecurity(items, checked, notChecked, root);
+    console.log(report);
+    if (cmd.out) {
+        mkdirSync(dirname(resolve(cmd.out)), { recursive: true });
+        writeFileSync(resolve(cmd.out), `${report}\n`, 'utf8');
+        console.log(`\nWritten to ${cmd.out}`);
+    }
+    return items;
+}

package/dist/engine/secure/triage.js

@@ -0,0 +1,133 @@
+// Phase 9: the part scanners do not do. A scanner says "this package has a CVE" or "this looks like
+// an AWS key"; a person then spends an afternoon deciding which of those matter. This decides, the
+// same way the rest of Tewip decides: deterministic checks first, evidence with every verdict, and
+// a plain next step.
+//
+// The verdicts are deliberately few:
+//   real-risk      something a person should act on now, with the evidence that makes it real;
+//   not-reachable  true of the repository, but not of what it runs: a dev-only dependency, or a
+//                  package nothing imports;
+//   noise          a shape without a secret behind it: documentation, a fixture, a placeholder;
+//   accepted       a person already looked and said so.
+//
+// A secret is never printed, only located. An "accepted" verdict is never invented by Tewip - it
+// comes from a file a person wrote.
+import { existsSync, readFileSync } from 'node:fs';
+import { join } from 'node:path';
+import { spawnSync } from 'node:child_process';
+/** Findings a person has already judged, one per line: "<kind> <where> - reason". */
+export function readAccepted(root) {
+    const file = join(root, '.tewip', 'security-accepted.txt');
+    if (!existsSync(file))
+        return new Set();
+    return new Set(readFileSync(file, 'utf8')
+        .split('\n')
+        .map((l) => l.split(' - ')[0].trim())
+        .filter(Boolean));
+}
+/** Whether git tracks this file. A secret in an ignored file never left the machine. */
+export function isTracked(root, file) {
+    const r = spawnSync(`git ls-files --error-unmatch "${file}"`, { cwd: root, shell: true, encoding: 'utf8', timeout: 30_000 });
+    return r.status === 0;
+}
+const SECRET_SEVERITY = {
+    'aws-secret-key': 'critical',
+    'aws-access-key-id': 'critical',
+    'private-key': 'critical',
+    'stripe-key': 'critical',
+    'github-token': 'high',
+    'gitlab-token': 'high',
+    'npm-token': 'high',
+    'openai-key': 'high',
+    'anthropic-key': 'high',
+    'slack-token': 'high',
+    'connection-string': 'high',
+    'google-api-key': 'medium',
+    'jwt': 'medium',
+};
+export function triageSecret(finding, root, accepted) {
+    const where = `${finding.file}:${finding.line}`;
+    const key = `secret ${where}`;
+    const base = {
+        kind: 'secret',
+        title: `${finding.what} in ${where}`,
+        severity: SECRET_SEVERITY[finding.kind] ?? 'medium',
+        where,
+        source: 'tewip secret scan',
+    };
+    if (accepted.has(key))
+        return { ...base, verdict: 'accepted', why: 'a person marked this one accepted in .tewip/security-accepted.txt', next: 'nothing: it was reviewed' };
+    if (finding.placeholder) {
+        return { ...base, verdict: 'noise', severity: 'low', why: `${finding.placeholder}: ${finding.evidence}`, next: 'nothing: this is not a credential' };
+    }
+    const tracked = isTracked(root, finding.file);
+    if (!tracked) {
+        return {
+            ...base,
+            verdict: 'not-reachable',
+            severity: 'low',
+            why: `git does not track ${finding.file}, so this never left the machine: ${finding.evidence}`,
+            next: `keep it that way: make sure ${finding.file} stays in .gitignore`,
+        };
+    }
+    return {
+        ...base,
+        verdict: 'real-risk',
+        why: `it is committed, and the value has the randomness of a real credential (${finding.entropy} bits per character): ${finding.evidence}`,
+        next: `revoke it at the issuer first, then remove it from the file and from the history. Changing the file alone is not enough: the old value stays in every clone.`,
+    };
+}
+/** Does anything in this project actually import the package? */
+export function isImported(root, pkg, sources) {
+    const name = pkg.replace(/^npm:/, '');
+    const patterns = [`from '${name}'`, `from "${name}"`, `require('${name}')`, `require("${name}")`, `import ${name}`, `import '${name}'`, `from ${name} import`];
+    for (const file of sources) {
+        let text;
+        try {
+            text = readFileSync(file, 'utf8');
+        }
+        catch {
+            continue;
+        }
+        if (patterns.some((p) => text.includes(p)))
+            return true;
+    }
+    return false;
+}
+const DEP_SEVERITY = { critical: 'critical', high: 'high', moderate: 'medium', medium: 'medium', low: 'low' };
+export function triageDependency(finding, root, sources, accepted) {
+    const where = `${finding.package}@${finding.version}`;
+    const base = {
+        kind: 'dependency',
+        title: `${finding.title} (${where})`,
+        severity: DEP_SEVERITY[finding.severity.toLowerCase()] ?? 'medium',
+        where,
+        source: finding.source,
+    };
+    const upgrade = finding.fixedIn ? `upgrade ${finding.package} to ${finding.fixedIn}` : `no fixed version is published yet: watch ${finding.id || 'the advisory'} and consider replacing the package`;
+    if (accepted.has(`dependency ${finding.package}`) || accepted.has(`dependency ${where}`)) {
+        return { ...base, verdict: 'accepted', why: 'a person marked this one accepted in .tewip/security-accepted.txt', next: 'nothing: it was reviewed' };
+    }
+    if (finding.dev) {
+        return {
+            ...base,
+            verdict: 'not-reachable',
+            severity: base.severity === 'critical' ? 'medium' : 'low',
+            why: 'it is a development dependency, so it is not in what users run',
+            next: `${upgrade} when convenient; it does not block a release`,
+        };
+    }
+    if (sources.length > 0 && !isImported(root, finding.package, sources)) {
+        return {
+            ...base,
+            verdict: 'not-reachable',
+            severity: base.severity === 'critical' ? 'medium' : 'low',
+            why: `nothing in this project imports ${finding.package}: it is pulled in by another package, so the vulnerable code may never run here`,
+            next: `${upgrade}, or check which dependency requires it (npm why ${finding.package}) before spending time on it`,
+        };
+    }
+    return { ...base, verdict: 'real-risk', why: `this project imports ${finding.package} directly, and the advisory applies to the installed version`, next: upgrade };
+}
+const ORDER = { 'real-risk': 0, 'not-reachable': 1, noise: 2, accepted: 3 };
+const SEV = { critical: 0, high: 1, medium: 2, low: 3 };
+export const worstFirst = (a, b) => ORDER[a.verdict] - ORDER[b.verdict] || SEV[a.severity] - SEV[b.severity];

package/dist/engine/work/jira.js

@@ -8,19 +8,26 @@
 //
 // Settings: TEWIP_JIRA_URL (https://your-team.atlassian.net), TEWIP_JIRA_EMAIL, TEWIP_JIRA_TOKEN
 // (an API token from id.atlassian.com).
+import { criteriaSection, wikiToText } from './wiki-text.js';
 /** The configuration from the environment, or null when Jira is not set up. */
 export function jiraFromEnv() {
     const baseUrl = process.env.TEWIP_JIRA_URL?.trim().replace(/\/+$/, '');
-    const email = process.env.TEWIP_JIRA_EMAIL?.trim();
-    const token = process.env.TEWIP_JIRA_TOKEN?.trim();
-    return baseUrl && email && token ? { baseUrl, email, token } : null;
+    if (!baseUrl)
+        return null;
+    const email = process.env.TEWIP_JIRA_EMAIL?.trim() ?? '';
+    const token = process.env.TEWIP_JIRA_TOKEN?.trim() ?? '';
+    // A tracker that anyone can read - Apache's, Jenkins', many company ones - needs no account at
+    // all, and reading a ticket to plan from it is the commonest thing Tewip does with Jira.
+    return { baseUrl, email, token };
 }
 export function jiraHelp() {
     return [
-        'Jira is not set up. Three settings, all from your own account:',
-        '  TEWIP_JIRA_URL=https://your-team.atlassian.net',
+        'Jira is not set up. One setting to read a public tracker:',
+        '  TEWIP_JIRA_URL=https://issues.apache.org/jira      (or your own Jira, Cloud or Server)',
+        'Two more to let Tewip write test cases back into your own project:',
         '  TEWIP_JIRA_EMAIL=you@company.com',
-        '  TEWIP_JIRA_TOKEN=...   (create one at id.atlassian.com/manage-profile/security/api-tokens)',
+        '  TEWIP_JIRA_TOKEN=...   (Cloud: id.atlassian.com/manage-profile/security/api-tokens;',
+        '                          Server or Data Center: your profile > Personal Access Tokens)',
     ].join('\n');
 }
 /** Atlassian's document format is a tree of nodes; this takes the words out of it. */
@@ -38,54 +45,95 @@ export function plainText(node) {
 const FIGMA_LINK = /https:\/\/(?:www\.)?figma\.com\/[^\s)"']+/g;
 /** Teams keep acceptance criteria under many headings; these are the common ones. */
 const CRITERIA_HEADING = /(?:^|\n)\s*(?:acceptance criteria|ac|definition of done|dod)\s*:?\s*\n/i;
+/**
+ * A ticket as Tewip reads it, from the raw fields. Kept apart from the client so it can be checked
+ * against real tickets without a network or an account.
+ */
+export function readIssue(raw) {
+    const fields = raw.fields;
+    // Jira Cloud sends a document tree; Server and Data Center send wiki markup in a string.
+    const description = (typeof fields.description === 'string' ? wikiToText(fields.description) : plainText(fields.description)).trim();
+    // Some projects have a custom field for acceptance criteria; others put a heading in the body.
+    const custom = Object.entries(fields).find(([name, value]) => /customfield/.test(name) && value && /acceptance/i.test(JSON.stringify(value).slice(0, 200)));
+    const fromHeading = criteriaSection(description);
+    return {
+        key: raw.key,
+        summary: String(fields.summary ?? ''),
+        description,
+        acceptanceCriteria: fromHeading || (custom ? plainText(custom[1]).trim() : ''),
+        type: String((fields.issuetype ?? {}).name ?? ''),
+        status: String((fields.status ?? {}).name ?? ''),
+        labels: fields.labels ?? [],
+        designLinks: [...new Set(`${description} ${JSON.stringify(fields.attachment ?? '')}`.match(FIGMA_LINK) ?? [])],
+    };
+}
 export class Jira {
     config;
+    /**
+     * Cloud speaks /rest/api/3 and wants a document tree; Server and Data Center speak /rest/api/2
+     * and want plain wiki markup. Most companies run the second kind, so both are supported: the
+     * version is guessed from the address and corrected the first time the other one answers.
+     */
+    version;
     constructor(config) {
         this.config = config;
+        this.version = /atlassian\.net$/i.test(new URL(config.baseUrl).hostname) ? 3 : 2;
+    }
+    /** Whether this site turned out to be Cloud, for anything that has to send a description. */
+    get isCloud() {
+        return this.version === 3;
     }
     async request(method, path, body) {
-        const auth = Buffer.from(`${this.config.email}:${this.config.token}`).toString('base64');
-        const res = await fetch(`${this.config.baseUrl}${path}`, {
+        // No credentials means an anonymous read: the header is left off entirely, because an empty
+        // one is rejected where no header at all is allowed.
+        const auth = this.config.token ? { Authorization: `Basic ${Buffer.from(`${this.config.email}:${this.config.token}`).toString('base64')}` } : {};
+        const send = async (version) => fetch(`${this.config.baseUrl}/rest/api/${version}${path}`, {
             method,
-            headers: { Authorization: `Basic ${auth}`, Accept: 'application/json', ...(body ? { 'Content-Type': 'application/json' } : {}) },
-            body: body ? JSON.stringify(body) : undefined,
+            headers: { ...auth, Accept: 'application/json', ...(body ? { 'Content-Type': 'application/json' } : {}) },
+            body: body ? JSON.stringify(this.bodyFor(version, body)) : undefined,
         });
+        let res = await send(this.version);
+        // The wrong guess answers 404 (no such endpoint) or 410 (gone); the other version is then tried
+        // once and remembered, so a Server site does not pay for the guess on every call.
+        if ((res.status === 404 || res.status === 410) && method === 'GET') {
+            const other = this.version === 3 ? 2 : 3;
+            const second = await send(other);
+            if (second.ok) {
+                this.version = other;
+                res = second;
+            }
+        }
         if (!res.ok)
-            throw new Error(`Jira ${method} ${path} -> ${res.status}: ${(await res.text()).slice(0, 300)}`);
+            throw new Error(`Jira ${method} /rest/api/${this.version}${path} -> ${res.status}: ${(await res.text()).slice(0, 300)}`);
         return (res.status === 204 ? undefined : await res.json());
     }
+    /** A description is a document tree on Cloud and a string of wiki markup everywhere else. */
+    bodyFor(version, body) {
+        const fields = body.fields;
+        if (!fields || typeof fields.description !== 'string')
+            return body;
+        if (version === 2)
+            return body;
+        return { ...body, fields: { ...fields, description: { type: 'doc', version: 1, content: [{ type: 'paragraph', content: [{ type: 'text', text: String(fields.description).slice(0, 30_000) }] }] } } };
+    }
     /** One ticket, flattened into the facts a plan can be built from. */
     async issue(key) {
-        const raw = await this.request('GET', `/rest/api/3/issue/${encodeURIComponent(key)}`);
-        const fields = raw.fields;
-        const description = plainText(fields.description).trim();
-        // Some projects have a custom field for acceptance criteria; others put a heading in the body.
-        const custom = Object.entries(fields).find(([name, value]) => /customfield/.test(name) && value && /acceptance/i.test(JSON.stringify(value).slice(0, 200)));
-        const fromHeading = CRITERIA_HEADING.test(description) ? description.split(CRITERIA_HEADING).slice(1).join('\n').trim() : '';
-        return {
-            key: raw.key,
-            summary: String(fields.summary ?? ''),
-            description,
-            acceptanceCriteria: fromHeading || (custom ? plainText(custom[1]).trim() : ''),
-            type: String((fields.issuetype ?? {}).name ?? ''),
-            status: String((fields.status ?? {}).name ?? ''),
-            labels: fields.labels ?? [],
-            designLinks: [...new Set(`${description} ${JSON.stringify(fields.attachment ?? '')}`.match(FIGMA_LINK) ?? [])],
-        };
+        const raw = await this.request('GET', `/issue/${encodeURIComponent(key)}`);
+        return readIssue(raw);
     }
     /** The issue types a project actually has, so Tewip can pick one that exists rather than guess. */
     async issueTypes(projectKey) {
-        const meta = await this.request('GET', `/rest/api/3/issue/createmeta?projectKeys=${encodeURIComponent(projectKey)}`);
+        const meta = await this.request('GET', `/issue/createmeta?projectKeys=${encodeURIComponent(projectKey)}&expand=projects.issuetypes`);
         return (meta.projects?.[0]?.issuetypes ?? []).map((t) => t.name);
     }
     /** Creates one issue and returns its key. Nothing else about it is ever changed afterwards. */
     async createIssue(input) {
-        const created = await this.request('POST', '/rest/api/3/issue', {
+        const created = await this.request('POST', '/issue', {
             fields: {
                 project: { key: input.projectKey },
                 issuetype: { name: input.type },
                 summary: input.summary.slice(0, 250),
-                description: { type: 'doc', version: 1, content: [{ type: 'paragraph', content: [{ type: 'text', text: input.description.slice(0, 30_000) }] }] },
+                description: input.description.slice(0, 30_000),
                 ...(input.labels?.length ? { labels: input.labels } : {}),
                 ...(input.parentKey ? { parent: { key: input.parentKey } } : {}),
             },
@@ -94,12 +142,12 @@ export class Jira {
     }
     /** Links a created issue to the ticket it came from, the only relationship Tewip makes. */
     async link(fromKey, toKey, type = 'Relates') {
-        await this.request('POST', '/rest/api/3/issueLink', { type: { name: type }, inwardIssue: { key: fromKey }, outwardIssue: { key: toKey } });
+        await this.request('POST', '/issueLink', { type: { name: type }, inwardIssue: { key: fromKey }, outwardIssue: { key: toKey } });
     }
     /** Issues Tewip created before, so the same scenario is never filed twice. */
     async findByLabel(projectKey, label) {
         const jql = `project = ${projectKey} AND labels = ${label} ORDER BY created DESC`;
-        const found = await this.request('GET', `/rest/api/3/search?jql=${encodeURIComponent(jql)}&maxResults=100&fields=summary`);
+        const found = await this.request('GET', `/search?jql=${encodeURIComponent(jql)}&maxResults=100&fields=summary`);
         return (found.issues ?? []).map((i) => ({ key: i.key, summary: i.fields.summary }));
     }
 }

package/dist/engine/work/wiki-text.js

@@ -0,0 +1,74 @@
+// Jira Cloud stores a description as a tree (Atlassian Document Format). Jira Server and Data
+// Center - which is what most companies still run, and what the public Apache and Jenkins trackers
+// use - store it as one string of wiki markup. Tewip has to read both, or half the tickets in the
+// world come out full of `h2.`, `{code}` and `[~someone]`.
+//
+// Measured on 150 real tickets from those two trackers: before this, 75 of 150 descriptions came
+// out as readable text.
+/** A character no ticket contains, holding a heading's place while lists are converted. */
+const MARK = String.fromCharCode(1);
+/** A heading, so the section it starts can be found afterwards: kept as "## Words". */
+const HEADING = /^\s*h([1-6])\.\s*(.+)$/gm;
+/** Decorations around words: *bold*, _italic_, +inserted+, -struck-, ??cited??, ^sup^, ~sub~. */
+const stripDecorations = (text) => text
+    .replace(/\{\{([^}]*)\}\}/g, '$1')
+    .replace(/(^|[\s(])([*_+])(?=\S)([^\n]*?\S)\2(?=[\s.,;:)!?]|$)/g, '$1$3')
+    .replace(/(^|[\s(])-(?=\S)([^\n]*?\S)-(?=[\s.,;:)!?]|$)/g, '$1$2')
+    .replace(/\?\?([^?]+)\?\?/g, '$1');
+export function wikiToText(markup) {
+    let text = markup.replace(/\r\n?/g, '\n');
+    // Blocks first: their contents are kept, their fences are not.
+    text = text
+        .replace(/\{code(?::[^}]*)?\}\n?([\s\S]*?)\{code\}/g, (_m, body) => `\n${body.trim()}\n`)
+        .replace(/\{noformat\}\n?([\s\S]*?)\{noformat\}/g, (_m, body) => `\n${body.trim()}\n`)
+        .replace(/\{quote\}\n?([\s\S]*?)\{quote\}/g, (_m, body) => `\n${body.trim()}\n`)
+        .replace(/\{panel(?::[^}]*)?\}\n?([\s\S]*?)\{panel\}/g, (_m, body) => `\n${body.trim()}\n`)
+        .replace(/\{color(?::[^}]*)?\}/g, '')
+        .replace(/\{anchor:[^}]*\}/g, '')
+        .replace(/\{[a-z-]+(?::[^}]*)?\}/g, ''); // any other macro fence
+    // Headings become "## Words", which is how the rest of Tewip recognises a section. They are
+    // marked first and written out last: in wiki markup a numbered list item also starts with "#",
+    // and converting headings straight to "##" turned every heading into a bullet.
+    text = text.replace(HEADING, (_m, level, words) => `\n${MARK}${level} ${stripDecorations(words).trim().replace(/[:\s]+$/, '')}`);
+    text = text
+        .replace(/\[([^\]|]+)\|([^\]]+)\]/g, '$1 ($2)') // [text|https://...]
+        .replace(/\[~([\w.@-]+)\]/g, '@$1') // a person
+        .replace(/\[([^\]|]{1,80})\]/g, '$1') // [SOME-123]
+        .replace(/^\s*\|\|(.+)\|\|\s*$/gm, (_m, row) => row.split('||').map((c) => c.trim()).filter(Boolean).join(' | '))
+        .replace(/^\s*\|(.+)\|\s*$/gm, (_m, row) => row.split('|').map((c) => c.trim()).filter(Boolean).join(' | '))
+        .replace(/^\s*[*#]{1,4}\s+/gm, '- ') // bullet and numbered lists
+        .replace(/\\\\\s*$/gm, '') // a forced line break
+        .replace(/^\s*----+\s*$/gm, '');
+    // The headings, written out now that nothing else can be mistaken for them.
+    text = text.replace(new RegExp(`${MARK}([1-6]) `, 'g'), (_m, level) => `${'#'.repeat(Number(level))} `);
+    return stripDecorations(text)
+        .split('\n')
+        .map((line) => line.replace(/\s+$/, ''))
+        .join('\n')
+        .replace(/\n{3,}/g, '\n\n')
+        .trim();
+}
+/**
+ * The heading that starts the acceptance criteria. Real tickets write it as "h2. Acceptance
+ * Criteria", "h1. +*Acceptance criteria*+", "*Acceptance criteria (Given / When / Then):*",
+ * "Suggested acceptance criteria" and "Definition of Done" - all of these are from the corpus.
+ */
+const CRITERIA_LINE = /^\s*(?:#{1,6}\s*)?(?:suggested|proposed|draft)?\s*(?:acceptance\s+criteria|acceptance|definition\s+of\s+done|dod)\b[^\n]*$/i;
+/** A line that starts a different section, which ends the criteria. */
+const OTHER_HEADING = /^#{1,6}\s+\S/;
+/** The acceptance criteria written in a description, or '' when the author wrote none. */
+export function criteriaSection(text) {
+    const lines = text.split('\n');
+    const start = lines.findIndex((line) => CRITERIA_LINE.test(line) && /acceptance|definition\s+of\s+done|dod/i.test(line));
+    if (start < 0)
+        return '';
+    const body = [];
+    for (let i = start + 1; i < lines.length; i++) {
+        if (OTHER_HEADING.test(lines[i]))
+            break;
+        body.push(lines[i]);
+    }
+    const section = body.join('\n').trim();
+    // A heading with nothing under it is not criteria; nor is one whose "criteria" is a single word.
+    return section.length > 10 ? section : '';
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "tewip-ai",
-  "version": "0.6.0",
+  "version": "0.7.0",
   "description": "Tewip finds why each failing test failed - real bug, flaky test, environment, test data or changed selector - for any stack, from the command line, CI, or your editor.",
   "keywords": [
     "test-failures",