@activate-core/test-forge 1.3.0 → 2.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (41) hide show
  1. package/README.md +67 -28
  2. package/dist/cli/index.js +39 -33
  3. package/dist/core/docs/cache.js +2 -1
  4. package/dist/core/docs/discover.js +49 -2
  5. package/dist/core/docs/doc-guidance.js +70 -29
  6. package/dist/core/docs/excerpts.js +294 -0
  7. package/dist/core/docs/grounding.js +55 -22
  8. package/dist/core/docs/reference-text.js +78 -0
  9. package/dist/core/docs/reference.js +60 -3
  10. package/dist/core/docs/review.js +38 -0
  11. package/dist/core/docs/source-of-truth.js +1 -1
  12. package/dist/core/docs/tickets.js +3 -3
  13. package/dist/core/gates/coverage.js +1 -16
  14. package/dist/core/gates/test-quality.js +295 -0
  15. package/dist/core/gates/validate.js +171 -7
  16. package/dist/core/generate/ai-generator.js +267 -117
  17. package/dist/core/generate/ai-usage.js +116 -5
  18. package/dist/core/generate/bind.js +91 -0
  19. package/dist/core/generate/complete-file.js +77 -0
  20. package/dist/core/generate/defect-review.js +59 -9
  21. package/dist/core/generate/doc-diff.js +140 -0
  22. package/dist/core/generate/harness.js +18 -34
  23. package/dist/core/generate/journey-memory.js +394 -0
  24. package/dist/core/generate/plan-cache.js +114 -7
  25. package/dist/core/generate/review-journey.js +3 -3
  26. package/dist/core/generate/run-generate.js +240 -82
  27. package/dist/core/generate/suite-planner.js +128 -23
  28. package/dist/core/generate/test-blocks.js +84 -0
  29. package/dist/core/generate/update-journey.js +118 -0
  30. package/dist/core/paths.js +90 -13
  31. package/dist/core/project/config.js +25 -2
  32. package/dist/core/project/env-template.js +2 -1
  33. package/dist/core/run/ci.js +1 -1
  34. package/dist/core/run/clean.js +1 -1
  35. package/dist/core/run/readiness.js +146 -0
  36. package/dist/core/run/run-tests.js +26 -2
  37. package/package.json +1 -1
  38. package/reporters/_shared.cjs +2 -2
  39. package/reporters/run-data.cjs +1 -1
  40. package/reporters/ui-helpers.cjs +78 -1
  41. package/dist/core/generate/verify-journey.js +0 -215
package/README.md CHANGED
@@ -94,7 +94,7 @@ never touched.
94
94
  | Command | What it does |
95
95
  | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
96
96
  | `testforge setup` | One-time setup: runs init, map, and discover in order, with progress. |
97
- | `testforge generate` | Generates a test file and a seed file for each journey doc. Only changed docs are regenerated. |
97
+ | `testforge generate` | Writes the tests for each journey doc. A changed doc updates only the tests of the rules that changed. Writes only: it runs nothing. |
98
98
  | `testforge update` | Refreshes the map, docs and tests when the **source code** or the **reference** has changed. Skips everything if neither has (no AI calls). Broad by design — for one journey, see below. |
99
99
 
100
100
  ### Advanced (run a single step)
@@ -105,7 +105,9 @@ never touched.
105
105
  | `testforge map` | Rebuild the project map (`testforge.md`) only. |
106
106
  | `testforge discover` | Redraft the per-journey docs in `docs/` only. |
107
107
  | `testforge env` | Rebuild `.env.testing.example` from the docs only. |
108
- | `testforge run` | Run the generated tests (`--journey` / `--area` / `--env-file`). |
108
+ | `testforge run` | Run the generated tests (`--journey` / `--area` / `--env-file`). Files whose feature is not in the code yet are held back as pending (`--all` runs them). |
109
+ | `testforge bind` | After a feature ships: which test files the code is ready for, which wait for the feature, which use a name the code spells differently. `--accept "doc=code"` takes the code's wording. No AI. |
110
+ | `testforge review` | Ask an AI reviewer whether the last failures are defects, test mistakes or the environment. On demand only. |
109
111
  | `testforge report` | Publish results to the dashboard — `--results` after a run, `--inventory` to register every generated check. |
110
112
 
111
113
  ### Flags worth knowing
@@ -115,6 +117,9 @@ never touched.
115
117
  | `--journey <name>` | discover, generate, run | Work on one journey instead of all of them. |
116
118
  | `--area <name>` | discover, generate, run | Work on one area (api, web, ops…). |
117
119
  | `--force` | generate | Regenerate even when the doc has not changed. |
120
+ | `--plan-only` | generate | Plan and check each journey (`testforge/memory/<area>/<id>/plan-check.md`), write no tests. |
121
+ | `--refresh` | discover | Fetch the reference again instead of the copy kept from the last run. |
122
+ | `--all` | run | Also run files whose feature is not in the code yet. |
118
123
  | `--refresh-docs` | update | Re-draft the docs even when nothing detectably changed — see below. |
119
124
  | `--reference <file\|url>` | setup | Base the docs on a spec instead of the code map. |
120
125
  | `--source map` | setup | Base the docs on `testforge.md`, no prompt. |
@@ -198,7 +203,7 @@ _expected_ behavior — a spec or requirements doc in the repo, a PDF, a
198
203
  website, or a Notion page. testforge then:
199
204
 
200
205
  1. reads the reference and compares it against `testforge.md`,
201
- 2. writes a full comparison to **`testforge.diff.md`** and prints a short
206
+ 2. writes a full comparison to **`testforge/reference/diff.md`** and prints a short
202
207
  summary of the differences,
203
208
  3. asks you to confirm, and on confirmation **drafts the docs from the
204
209
  reference** — expected behavior comes from the reference, while routes,
@@ -207,9 +212,9 @@ website, or a Notion page. testforge then:
207
212
 
208
213
  The reference becomes the source of truth — expected behavior from a human
209
214
  or client, not just what the code happens to do. The choice is remembered
210
- in `test-artifacts/source.json`, so a later `testforge update` refreshes the
215
+ in `testforge/source.json`, so a later `testforge update` refreshes the
211
216
  docs from the same source without asking again. Your review of
212
- `testforge.diff.md` stays as an audit of what differed and why.
217
+ `testforge/reference/diff.md` stays as an audit of what differed and why.
213
218
 
214
219
  **Non-interactive use (CI, scripts):**
215
220
 
@@ -357,29 +362,63 @@ Test results are kept as a workflow artifact for 14 days.
357
362
  reported as a pending decision, never as a defect. Keeps the reference
358
363
  you share with a client clean while the tests still know what is undecided.
359
364
 
360
- ## How `generate` checks its own work
361
-
362
- Every test is run against the real app before it is handed over, so a
363
- mistake in a generated test does not reach your team as a product defect:
364
-
365
- 1. **Each file on its own.** A test that fails because of its own mistake
366
- (a wrong locator, a field the server never sends, a bad precondition) is
367
- repaired from the runner's output. A guard refuses any repair that
368
- deletes a test, adds a skip or removes an assertion.
369
- 2. **The whole journey together**, the way CI runs it. Files that pass alone
370
- but break each other (one signs in again as a shared account and ends
371
- another's session) are repaired with that context.
372
- 3. **An independent review of every claimed defect.** A separate reviewer
373
- starts from "not proven" and must find the application code at fault.
374
- `generate` then lists, one line per root cause:
375
- - **defects**: a [spec] requirement the app breaks;
376
- - **findings**: real behaviour against a [baseline]/[observed] criterion;
377
- - **pending decisions**: covered by `decisions`.
378
- A claim it rejects is repaired once more. Anything still unexplained is
379
- reported as unverified, with the advice not to report it.
380
-
381
- Each run is time-limited (`verifyTimeoutMs`, per file). On expiry the
382
- runner stops with everything it started.
365
+ ## How `generate` keeps the AI cost down
366
+
367
+ The journey doc is read once and kept as the journey's **memory**
368
+ (`testforge/memory/<area>/<id>/`): its rules with their source, level, tests and
369
+ history, the plan, and the code facts. No later step sends the whole doc again.
370
+
371
+ 1. **Rules.** The doc is split into rules (R1, R2 …) without AI. Rule ids stay
372
+ stable when the doc changes. Every rule records its source, the files that
373
+ test it, why it was skipped, and its earlier wordings.
374
+ 2. **Plan, without AI.** Each numbered section of the doc becomes one test file
375
+ (`"planner": "ai"` asks the AI instead). Rules another journey already tests
376
+ are not planned again. The plan check (`plan-check.md`) lists every rule that
377
+ gets tests, every skip with its reason, and any **must** rule left out.
378
+ 3. **Code facts, once.** The source the doc points at (the files it cites, the
379
+ components its controls are named after, the pages and hooks that reach
380
+ them) is collected without AI and summarised in one call.
381
+ 4. **Writing.** Each call writes a few files (`filesPerCall`, default 3) and
382
+ gets only their rules, their controls' facts and the testkit's exports; the
383
+ shared rules for writing a test are a cached system prompt. Tests find every
384
+ element with `kit.find(page, kind, name)`, check outcomes rather than markup,
385
+ and stub a state they cannot create.
386
+ 5. **Checks before anything runs.** A file is refused, and written once more
387
+ with the reason, if it sleeps for a fixed time, skips because an element is
388
+ missing, has no test or no assertion, spends more than `maxTestsPerRule`
389
+ tests on one rule, or names a label found neither in the code nor the doc.
390
+ 6. **Every rule, a test that can fail.** Each test is checked on its own:
391
+ every planned rule has a test, or a `// cannot-test: R7 — <the state this
392
+ area cannot set up>` line ("the locator is unknown" is not such a state);
393
+ no assertion accepts the good and the bad outcome alike, compares the page's
394
+ whole text, sits only inside an `if`, or repeats what its own stub returned;
395
+ `kit.find` is awaited and finds elements by the names the screen shows;
396
+ `kit.openStep` names a step the testkit has. The tests that fail a check are
397
+ taken out and their rules written again in ONE small call; what is still
398
+ wrong is removed and reported. The summary line says how many planned rules
399
+ have a test, which cannot be tested here and why, and which have none, and
400
+ `rules.json` / `plan-check.md` record the same.
401
+ 7. **A changed doc.** Only what changed is touched: a renamed label is replaced
402
+ in its tests, a removed rule's tests are deleted (both without AI); a
403
+ reworded or new rule gets its tests written by one small call.
404
+
405
+ **A feature that is not live yet.** Put `Status: not built` or
406
+ `Status: behind flag <name>` in the journey doc's header. Every file of the
407
+ journey is stamped `// testforge-pending: …`, and `testforge run` (and `bind`)
408
+ hold it back until the line is removed; `--all` runs it anyway.
409
+
410
+ `generate` runs nothing. `testforge run` runs the suite; `testforge review`
411
+ judges failures when you ask for it.
412
+
413
+ Every Claude call reports its real token use and cost; `TESTFORGE_TRACE=1`
414
+ also records each call's prompt and events under `testforge/runs/trace/`.
415
+
416
+ ## Project state
417
+
418
+ Everything TestForge keeps lives in `testforge/`. Commit it, except `runs/`
419
+ (run output and AI logs): a teammate who pulls it starts from your memory, plans
420
+ and code facts instead of paying for them again. A project set up by an older
421
+ version is moved from `test-artifacts/` on the first command.
383
422
 
384
423
  ## Notes
385
424
 
package/dist/cli/index.js CHANGED
@@ -21,6 +21,8 @@
21
21
  import { VERSION } from '../core/index.js';
22
22
  import { runGenerate } from '../core/generate/run-generate.js';
23
23
  import { runReview } from '../core/generate/review-journey.js';
24
+ import { migrateState } from '../core/paths.js';
25
+ import { acceptNames, bindJourneys } from '../core/generate/bind.js';
24
26
  import { runClean } from '../core/run/clean.js';
25
27
  import { aiUsage, describeUsage } from '../core/generate/ai-usage.js';
26
28
  import { envDiff, describeEnvDiff } from '../core/project/env-diff.js';
@@ -62,6 +64,7 @@ function sourceOptions() {
62
64
  reference,
63
65
  source: flagValue('--source'),
64
66
  replace: process.argv.includes('--replace'),
67
+ refresh: process.argv.includes('--refresh'),
65
68
  };
66
69
  }
67
70
  /**
@@ -91,11 +94,11 @@ function printUsage() {
91
94
  console.log(' testforge generate --basic Use the basic generator only');
92
95
  console.log(' testforge generate --force Regenerate everything');
93
96
  console.log(' testforge generate --no-wait Stop when the AI hits its usage limit (default: wait for the reset and resume)');
94
- console.log(' testforge generate --no-verify Skip running each journey once after generating (default: run and repair)');
97
+ console.log(' testforge generate --plan-only Plan and check each journey (testforge/memory/<area>/<id>/plan-check.md), write no tests');
95
98
  console.log(' testforge generate --journey <name> Generate just ONE journey');
96
99
  console.log(' testforge generate --area <area> Generate a whole area (e.g. api)');
97
100
  console.log(' testforge discover Draft journey docs (asks: map or reference)');
98
- console.log(' testforge discover --journey <name> Draft just ONE journey doc');
101
+ console.log(' testforge discover --journey <name> Draft just ONE journey doc', ' testforge discover --journey <name> --reference <x> [--refresh] One journey from a reference: the reference is read once and kept (--refresh reads it again)');
99
102
  console.log(' testforge discover --reference <file|url> Base docs on a reference');
100
103
  console.log(' testforge discover --tickets <folder> Base docs on exported tickets (one file per ticket); each source names its ticket');
101
104
  console.log(' (a reference only writes the journeys it');
@@ -107,6 +110,7 @@ function printUsage() {
107
110
  console.log(' testforge env Write .env.testing.example from docs');
108
111
  console.log(' testforge env --diff Only the variables the suite reads that the env file does not set');
109
112
  console.log(' testforge ci --github Write a GitHub Actions workflow that runs the suite against a deployed environment');
113
+ console.log(' testforge bind [--journey j] [--accept "doc=code"] Which tests the code is ready for, which wait for the feature, which use another name; --accept takes the code\'s wording (no AI)');
110
114
  console.log(' testforge review [--journey j] [--rerun] Judge the last failures again (defect / test fault / environment …), no repair');
111
115
  console.log(' testforge clean [--run <id> | --older-than <days>] [--yes] Remove what earlier runs created (dry run without --yes)');
112
116
  console.log(' testforge run [--area a] [--journey j] Run the generated tests (Jest/Playwright)');
@@ -115,6 +119,10 @@ function printUsage() {
115
119
  console.log(' testforge run -- <flags> Pass anything else straight to the runner');
116
120
  }
117
121
  async function main() {
122
+ // A project set up by an older version keeps its state under test-artifacts/: move it once.
123
+ const moved = command && command !== 'help' && command !== '--help' ? migrateState() : [];
124
+ if (moved.length)
125
+ console.log(`Moved TestForge's state into testforge/ (${moved.length} item(s)); nothing was rebuilt.\n`);
118
126
  if (command === 'update') {
119
127
  await runUpdate('.', { refreshDocs: process.argv.includes('--refresh-docs') });
120
128
  }
@@ -146,6 +154,32 @@ async function main() {
146
154
  const days = flagValue('--older-than');
147
155
  console.log(await runClean({ runId: flagValue('--run'), olderThanDays: days != null ? Number(days) : undefined, yes: process.argv.includes('--yes') }));
148
156
  }
157
+ else if (command === 'bind') {
158
+ // After a feature ships: which tests can run, which wait for the code, which use a different name.
159
+ const accept = process.argv.flatMap((a, k) => (process.argv[k - 1] === '--accept' ? [a] : []));
160
+ if (accept.length) {
161
+ const pairs = accept.map((a) => a.split('=')).filter((p) => p.length === 2 && p[0] && p[1]);
162
+ const notes = acceptNames(pairs, { journey: flagValue('--journey') });
163
+ for (const n of notes)
164
+ console.log(` ${n}`);
165
+ console.log(notes.length ? '\nRun `testforge generate` to record the rename in the journey\'s memory (no AI).' : 'Nothing used those names.');
166
+ }
167
+ else {
168
+ for (const r of bindJourneys({ journey: flagValue('--journey') })) {
169
+ const by = (s) => r.files.filter((f) => f.state === s);
170
+ console.log(` ${r.journey}: ${by('built').length} ready, ${by('partial').length} partly built, ${by('not-built').length} not built yet, ${by('renamed').length} with a different name`);
171
+ for (const f of r.files) {
172
+ if (f.state === 'built')
173
+ continue;
174
+ const what = f.state === 'renamed'
175
+ ? f.renamed.map((x) => `"${x.name}" → code has "${x.nearest}"`).join('; ')
176
+ : `not in the code: ${f.missing.slice(0, 4).map((m) => `"${m}"`).join(', ')}${f.missing.length > 4 ? ' …' : ''}`;
177
+ console.log(` ${f.state.padEnd(9)} ${f.file}: ${what}`);
178
+ }
179
+ }
180
+ console.log('\nA different name is not a defect: change the code to the doc, or accept the code\'s wording with --accept "doc wording=code wording".');
181
+ }
182
+ }
149
183
  else if (command === 'review') {
150
184
  console.log('Reviewing the last failures of each journey (no repair)...\n');
151
185
  const summaries = await runReview({ journey: flagValue('--journey'), rerun: process.argv.includes('--rerun') });
@@ -159,7 +193,7 @@ async function main() {
159
193
  else
160
194
  console.log(` ${r.journey}: ${r.failures} failure(s) → ` + Object.entries(r.verdicts).map(([k, n]) => `${n} ${k}`).join(', '));
161
195
  }
162
- console.log('\nVerdicts written to test-artifacts/defect-review/. The next run publishes them with each failed test.');
196
+ console.log('\nVerdicts written to testforge/runs/defect-review/. The next run publishes them with each failed test.');
163
197
  }
164
198
  else if (command === 'discover') {
165
199
  console.log('Drafting journey files with AI...');
@@ -189,6 +223,7 @@ async function main() {
189
223
  journey: flagValue('--journey'),
190
224
  envFile: flagValue('--env-file'),
191
225
  passThrough,
226
+ all: process.argv.includes('--all'),
192
227
  });
193
228
  process.exitCode = code;
194
229
  }
@@ -203,11 +238,7 @@ async function main() {
203
238
  : process.argv.includes('--wait-for-limit')
204
239
  ? true
205
240
  : undefined, // undefined -> config.waitForLimit (default on)
206
- verify: process.argv.includes('--no-verify')
207
- ? false
208
- : process.argv.includes('--verify')
209
- ? true
210
- : undefined, // undefined -> config.verify (default on)
241
+ planOnly: process.argv.includes('--plan-only'),
211
242
  });
212
243
  if (results.length === 0) {
213
244
  console.log('No journey files found in docs/. Add a .md file and try again.');
@@ -238,9 +269,6 @@ async function main() {
238
269
  for (const w of r.warnings ?? []) {
239
270
  console.log(` ! ${w}`);
240
271
  }
241
- if (r.verified === true)
242
- console.log(' verified: every file ran; every failure left was reviewed and is listed below');
243
- printReviewed(r.reviewed ?? []);
244
272
  }
245
273
  const generated = results.filter((r) => r.status !== 'skipped').length;
246
274
  // Rows with no test dir are notices (a non-journey file, the AI going
@@ -318,28 +346,6 @@ async function main() {
318
346
  printUsage();
319
347
  }
320
348
  }
321
- /** Reviewed failures, one line per root cause, strongest first. */
322
- function printReviewed(reviewed) {
323
- const sections = [
324
- ['defect', 'DEFECTS — a [spec] requirement the app breaks'],
325
- ['finding', 'FINDINGS — real app behaviour against a [baseline]/[observed] criterion (lower priority)'],
326
- ['pending-decision', 'PENDING DECISIONS — not requirements yet; take them to product, do not report as defects'],
327
- ];
328
- for (const [verdict, heading] of sections) {
329
- const rows = reviewed.filter((r) => r.verdict === verdict);
330
- if (!rows.length)
331
- continue;
332
- console.log(` ${heading}:`);
333
- for (const r of rows) {
334
- const where = r.tests.slice(0, 3).map((t) => `${basename(t.file)}:${t.line}`).join(', ');
335
- const more = r.tests.length > 3 ? ` +${r.tests.length - 3} more` : '';
336
- console.log(` - ${r.claim}`);
337
- console.log(` ${r.tests.length} test(s): ${where}${more}${r.evidence ? ` · app: ${r.evidence}` : ''}`);
338
- if (r.reason)
339
- console.log(` ${r.reason}`);
340
- }
341
- }
342
- }
343
349
  main().catch((err) => {
344
350
  console.error(err instanceof Error ? err.message : String(err));
345
351
  process.exit(1);
@@ -12,7 +12,7 @@
12
12
  import { createHash } from 'node:crypto';
13
13
  import { readFileSync, writeFileSync, existsSync, mkdirSync, readdirSync, statSync, } from 'node:fs';
14
14
  import { dirname, join, extname } from 'node:path';
15
- import { STATE_DIR, LEGACY_STATE_DIR } from '../paths.js';
15
+ import { STATE_DIR, LEGACY_STATE_DIR, OLD_STATE_DIR } from '../paths.js';
16
16
  /** Make a short fingerprint of a file's content. */
17
17
  export function hashContent(content) {
18
18
  return createHash('sha256').update(content).digest('hex');
@@ -66,6 +66,7 @@ const SKIP_DIRS = new Set([
66
66
  'docs',
67
67
  STATE_DIR,
68
68
  LEGACY_STATE_DIR,
69
+ OLD_STATE_DIR,
69
70
  'generated-tests',
70
71
  ]);
71
72
  export function hashProjectSource(rootDir) {
@@ -21,8 +21,10 @@ import { loadConfig, splitCommand, withEditTools } from '../project/config.js';
21
21
  import { DEEP_DOC_GUIDANCE } from './doc-guidance.js';
22
22
  import { promptConfirm } from '../prompt.js';
23
23
  import { loadSourceChoice, resolveDocsSource, saveJourneyReferences, saveSourceChoice, } from './source-of-truth.js';
24
- import { analyzeReference, draftDocsFromReference, DIFF_PATH, } from './reference.js';
25
- import { reviewDocs } from './review.js';
24
+ import { analyzeReference, buildJourneyDraftPrompt, draftDocsFromReference, draftJourneyDoc, DIFF_PATH, } from './reference.js';
25
+ import { buildLeanReviewPrompt, enforceSpecSources, reviewDocs } from './review.js';
26
+ import { readReference, sectionFor } from './reference-text.js';
27
+ import { sourceExcerpts } from './excerpts.js';
26
28
  import { runAiSync } from '../generate/ai-usage.js';
27
29
  /** Scope line: one journey, or all of them. */
28
30
  function scopeLines(journey) {
@@ -168,6 +170,12 @@ function snapshotDocs(docsDir) {
168
170
  */
169
171
  export async function runDiscover(srcDir, docsDir, opts = {}, journey) {
170
172
  const source = await resolveDocsSource(opts);
173
+ // One journey from a reference: no comparison of the whole product, the
174
+ // reference read once and cut to this journey, drafted and reviewed in one
175
+ // call each with no tools.
176
+ if (source.mode === 'reference' && source.reference && journey) {
177
+ return draftOneJourney(srcDir, docsDir, source.reference, journey, opts.refresh);
178
+ }
171
179
  if (source.mode === 'reference' && source.reference) {
172
180
  console.log(`\nComparing the reference against testforge.md: ${source.reference}`);
173
181
  console.log('(reading the reference and the map — this may take a moment)\n');
@@ -287,3 +295,42 @@ async function discoverFromCode(srcDir, docsDir, journey) {
287
295
  await reviewDocs(docsDir, changed, srcDir);
288
296
  return out;
289
297
  }
298
+ const docTitle = (text) => /^# Journey:\s*(.+)$/m.exec(text)?.[1]?.trim();
299
+ const kebab = (t) => t.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-|-$/g, '');
300
+ async function draftOneJourney(srcDir, docsDir, reference, journey, refresh = false) {
301
+ const config = loadConfig();
302
+ const text = readReference(reference, refresh);
303
+ const section = sectionFor(text, journey);
304
+ const excerpts = sourceExcerpts(section, [srcDir, ...(config.relatedDirs ?? [])]);
305
+ const docs = [...snapshotDocs(docsDir)].filter(([p, t]) => !notAJourneyDoc(p, t));
306
+ const existing = docs.find(([, t]) => kebab(docTitle(t) ?? '') === kebab(journey))?.[0];
307
+ const others = docs.filter(([p]) => p !== existing).map(([p, t]) => `${docTitle(t) ?? p} (${p})`);
308
+ const areas = Object.keys(config.testOutput ?? {}).length ? Object.keys(config.testOutput ?? {}) : ['web', 'api'];
309
+ const owner = loadSourceChoice()?.journeys?.[existing ?? ''];
310
+ if (existing && owner && owner !== reference) {
311
+ throw new Error(`${existing} comes from another reference (${owner}); not redrafting it from ${reference}.`);
312
+ }
313
+ console.log(`\nDrafting "${journey}" from the reference (${section.length.toLocaleString('en-GB')} characters of it, one call, no tools)...`);
314
+ const draft = draftJourneyDoc(buildJourneyDraftPrompt(journey, section, excerpts, areas, others));
315
+ if (!draft.markdown)
316
+ throw new Error('The AI answered without a journey doc.');
317
+ console.log('Reviewing it for missing scenarios (one call, no tools)...');
318
+ let doc = draft.markdown;
319
+ try {
320
+ const reviewed = draftJourneyDoc(buildLeanReviewPrompt(doc, section, excerpts, others));
321
+ if (reviewed.markdown)
322
+ doc = reviewed.markdown;
323
+ }
324
+ catch (e) {
325
+ console.log(` review failed, kept as drafted (${e.message.split('\n')[0]})`);
326
+ }
327
+ const { markdown, retagged } = enforceSpecSources(doc);
328
+ if (retagged)
329
+ console.log(` ${retagged} [spec] line(s) without a quoted source re-tagged [observed]`);
330
+ const rel = existing ?? join(draft.area && areas.includes(draft.area) ? draft.area : areas[0], `${kebab(docTitle(markdown) ?? journey)}.md`);
331
+ mkdirSync(join(docsDir, rel, '..'), { recursive: true });
332
+ writeFileSync(join(docsDir, rel), markdown, 'utf-8');
333
+ if (!existing)
334
+ saveJourneyReferences([rel], reference);
335
+ return `${existing ? 'Rewrote' : 'Wrote'} ${join(docsDir, rel)} from the reference. Please review it, then run \`testforge generate --journey ${kebab(journey)}\`.`;
336
+ }
@@ -45,30 +45,68 @@ export const DEEP_DOC_GUIDANCE = [
45
45
  `- "[spec]" — the source states this behaviour. A failure is a defect`,
46
46
  ` against the spec.`,
47
47
  `- "[baseline]" — universal robustness that holds for ANY well-built app.`,
48
- ` Add these YOURSELF for every field, control and action the journey`,
49
- ` has, even when the source says nothing about them:`,
48
+ ` The client describes what to build; it rarely says how it must NOT`,
49
+ ` break. That is your job: think like a tester who wants to break it and`,
50
+ ` like a user who is technical enough to find its loopholes. Add these`,
51
+ ` YOURSELF for every screen, form, list, action and endpoint the journey`,
52
+ ` has, even when the source says nothing about them. Write each one's`,
53
+ ` LEVEL and CATEGORY right after the tag, e.g. "[baseline] (must,`,
54
+ ` security)". Every "must" that fits the journey has to be there.`,
55
+ ` MUST — security:`,
56
+ ` * One user never reaches another user's data or actions: changing an id`,
57
+ ` in a URL, a request or a hidden field is refused (not "not found by`,
58
+ ` accident" — refused). A page or endpoint that needs a login is refused`,
59
+ ` without one; a role cannot use another role's actions.`,
60
+ ` * Whatever the client sends is not trusted: a price, an amount, a`,
61
+ ` discount, a role, an owner id or a status changed in the request is`,
62
+ ` refused or recomputed by the server.`,
63
+ ` * An HTML, script or injection payload in any input is stored and shown`,
64
+ ` as text, never executed, never a server error (5xx).`,
65
+ ` * Passwords are masked; passwords, tokens and codes never appear in a`,
66
+ ` URL, a query string or a log the user can see.`,
67
+ ` * Repeated wrong attempts (password, code, coupon, PIN) are limited and`,
68
+ ` the limit is told to the user.`,
69
+ ` MUST — money and data:`,
70
+ ` * Double submission: an action with a side effect (pay, book, order,`,
71
+ ` send, create) done`,
72
+ ` twice — a double click, a repeated Enter, two tabs, a retry after a`,
73
+ ` timeout — leaves exactly one record, one charge, one message.`,
74
+ ` * A flow that fails half-way leaves no half-made record and no money`,
75
+ ` taken without the thing bought; the user sees what happened.`,
76
+ ` * Amounts, totals, discounts and limits are the same on every screen and`,
77
+ ` in the server's answer.`,
78
+ ` MUST — concurrency:`,
79
+ ` * Two users or two tabs acting on the same thing at once (the same slot,`,
80
+ ` the last item, a code with limited uses, the same record edited) —`,
81
+ ` exactly one wins and the other gets a clear reason.`,
82
+ ` * Nothing hangs: every action ends within its timeout with a result or a`,
83
+ ` visible error; a request that never answers never leaves a spinner`,
84
+ ` forever or a control disabled for good.`,
85
+ ` MUST — error boundaries:`,
86
+ ` * Every load and every action, when the server answers 500, the network`,
87
+ ` drops, it times out, or the answer is empty or malformed: a visible`,
88
+ ` message, the page stays usable, no blank page, no crash, the user's`,
89
+ ` input is kept.`,
90
+ ` * One part of a page failing never takes the whole page down.`,
91
+ ` SHOULD — validation:`,
50
92
  ` * Inputs: empty and whitespace-only values; leading/trailing spaces;`,
51
- ` a very long value (5,000 characters); unicode and emoji; an HTML or`,
52
- ` script payload. The app never crashes, never answers with a server`,
53
- ` error (5xx), never executes the payload (it is shown as text), and`,
54
- ` either accepts the value or rejects it with a visible reason.`,
55
- ` * No silent failure: every rejected submit shows a visible reason; a`,
56
- ` server error, network failure or timeout shows a visible message`,
57
- ` and leaves the form usable.`,
58
- ` * Double submission: a double click or a repeated Enter sends at most`,
59
- ` one request with a side effect (one record, one charge, one`,
60
- ` message); the control is disabled or ignored while a request is`,
61
- ` pending.`,
62
- ` * Keyboard: the form can be filled and submitted with the keyboard`,
63
- ` alone; Enter submits.`,
64
- ` * Navigation: refresh, Back, and opening a mid-flow URL directly never`,
65
- ` crash or leave a blank, broken page.`,
66
- ` * Sensitive data: passwords are masked; passwords and tokens never`,
67
- ` appear in a URL or query string.`,
68
- ` * Access: a page or action that needs a login is not usable logged`,
69
- ` out; one user never sees another user's data.`,
70
- ` * Choice controls (dropdown, radio, checkbox, list): every option`,
71
- ` shown can be selected and submitted.`,
93
+ ` a very long value (5,000 characters); unicode and emoji. Each is`,
94
+ ` accepted or rejected with a visible reason, never a crash.`,
95
+ ` * No silent failure: every rejected submit says why, next to the field`,
96
+ ` or the form (a wrong password with no message is a defect).`,
97
+ ` * The screen and the server accept exactly the same values.`,
98
+ ` SHOULD — state and navigation:`,
99
+ ` * Refresh, Back, Forward and opening a mid-flow URL directly never crash`,
100
+ ` or leave a blank page; an expired session sends the user to log in and`,
101
+ ` back; after logging out nothing of the account is shown.`,
102
+ ` * Choice controls (dropdown, radio, checkbox, list): every option shown`,
103
+ ` can be selected and submitted; long lists page or scroll.`,
104
+ ` SHOULD — regression:`,
105
+ ` * Every defect the source or the doc already names keeps a test, so it`,
106
+ ` cannot quietly come back.`,
107
+ ` COULD — accessibility:`,
108
+ ` * The form can be filled and submitted with the keyboard alone; Enter`,
109
+ ` submits; pop-ups close with Escape; controls have readable names.`,
72
110
  ` [baseline] is ROBUSTNESS only. Never present product POLICY you`,
73
111
  ` invented (a maximum length, a password rule, an error wording, a`,
74
112
  ` limit) as [baseline].`,
@@ -89,15 +127,18 @@ export const DEEP_DOC_GUIDANCE = [
89
127
  ` becomes a test that fails on a correct product.`,
90
128
  ` An [observed] criterion is about what the customer sees or what the`,
91
129
  ` server accepts or refuses — never an implementation detail (a storage`,
92
- ` key name, a CSS class, an internal flag, a request field name) unless the`,
93
- ` source itself is about it.`,
94
- ` Only what a test can drive is a criterion. Behaviour no screen or`,
95
- ` endpoint can reach ("not tested through the screen") is not tagged —`,
96
- ` leave it out, or mention it in Test setup if the generator needs to know.`,
130
+ ` key name, a CSS class, an internal flag, a request field name, a log`,
131
+ ` line, a lock, a cache) unless the source itself is about it.`,
132
+ ` Only what a test in THIS area can drive is a criterion. Behaviour no`,
133
+ ` screen or endpoint of this area can reach (a backend job, another app,`,
134
+ ` an operator's action) is not tagged — list it under "## Not requirements"`,
135
+ ` with where it is checked, or mention it in Test setup if the generator`,
136
+ ` needs to know.`,
97
137
  ``,
98
138
  `METHOD. Work through these steps for every journey, in order:`,
99
139
  `1. CONTROLS. Start the doc with a "## Controls" table listing EVERY`,
100
- ` interactive element and where it appears: each input, button, link,`,
140
+ ` interactive element and where it appears, each row's first cell its id`,
141
+ ` C1, C2, C3 …: each input, button, link,`,
101
142
  ` checkbox, code box, dropdown, pop-up, menu item, and the page itself.`,
102
143
  ` Secondary controls count as much as the main form: Resend, Back,`,
103
144
  ` show/hide, close, apply, sign-out, provider buttons. Apply every`,