tosijs-ui 1.16.7 → 1.16.8

package/CHANGELOG.md

@@ -1,5 +1,66 @@
 # Changelog
 
+## 1.16.8
+
+A patch for honest results. The doc-test lane can no longer pass over pages that never
+reported, three more of tjs-lang's findings are fixed, and an example that only logs now
+shows what it logged. Two things to know on upgrade: `ts` examples lose tjs's runtime
+parameter guards (below), and one thing may newly go red: a doc-test
+run that was silently losing pages will now fail and name them. `dist/iife.js` grows by
+1.0 kB gzip (484.0 → 485.0 kB).
+
+### Three more from tjs-lang's sweep of its own site (#210)
+
+- **Inline tjs tests ran with plain-JavaScript semantics**, so passing tests were shown
+  failing: `"hello world".capitalize is not a function` for a local `extend`, `Expected false
+  but got true` for a boxed boolean. This release prepares the fix and tjs-lang delivers it:
+  an example's inline tests now use the runner `tjs()` itself returns, and from the tjs-lang
+  release after 0.14.0-rc.2 that runner is built from the rewritten test bodies. On 0.13.13,
+  which this release still pins, nothing changes.
+- **The `ts` dialect overrode converted TypeScript's semantics.** It passed `dialect: 'tjs'`
+  to `tjs()`, which outranks the annotation `fromTS` writes, so a TS example's
+  `new Calculator(…)` was refused as native TJS. It now passes no dialect. **This changes
+  what every `ts` example does at run time:** converted TypeScript now has JavaScript's
+  semantics, so the runtime parameter guards are gone too. `add('a', 2)` with
+  `(a: number, b: number)` returned a type error before and returns `'a2'` now, as it would
+  in TypeScript. `tjs` examples are unaffected.
+- **An example whose output is only `console.log` was an empty box.** When a run finishes with
+  nothing rendered into its preview, the lines it logged (and its inline-test summary, when
+  one was run) are shown where the preview would be. It steps aside when the example renders
+  anything, never writes into `preview`, and follows the existing console switches. Inline
+  examples only: an `:iframe` example's output stays in its Console tab.
+
+Not in this release: examples written as modules (`export const x = …`) still fail. Stripping
+`export` safely needs a JavaScript parser, which tosijs-ui does not ship; tjs-lang already
+bundles one, so the ask is with them to export their stripper.
+
+### The doc-test lane can no longer pass over pages that reported nothing (#2874)
+
+The background test runner counted a page as tested when it finished or timed out, whether or
+not any results arrived, and the lane passed on "0 failed". A run once passed with results for
+7 of 19 pages.
+
+- A page with test blocks that reports no tests, or does not finish within 45 seconds, is now
+  recorded as a failed test naming the page and the reason. A page that timed out after some
+  of its examples had reported was still recorded as a pass in the first cut of this fix; a
+  timeout now fails the page whatever had reported.
+- The parent waited 30 seconds per page, the same as the frame's own stall timeout, so a
+  stall report could arrive just after the parent moved on. The page budget is now 45s.
+- Which pages have tests is decided by parsing the markdown the way the page is rendered. A
+  ` ```test ` line shown inside a longer or `:static` block no longer makes a page count as
+  one with tests.
+- `bun run test-browser` now exits non-zero unless every page with tests has recorded results,
+  and prints how many pages its count covers.
+- The page a run starts on is tested in place, by its own code path. That path replaced the
+  page's results each time one example finished, so the last example was the whole page and
+  a failing earlier one could vanish; and it marked the page tested on a timer. It now adds
+  its examples up and waits for all of them, by the same rule the iframes use. A deployed
+  (non-localhost) page with tests enabled follows the same rule.
+- The runner accepted `tosi-test-results` messages from any window on the page; it now
+  requires its own test frame as the source.
+- `testReportVerdict` is exported from `tosijs-ui/site` for adopters' own gates. A gate that
+  checks only `failed === 0` should also require results for every page with tests.
+
 ## 1.16.7
 
 A patch for readers that are not browsers, and for builds that were too polite. Every doc page

package/dist/doc-browser.d.ts

@@ -1,5 +1,6 @@
 import { TestResults } from './live-example/test-harness.js';
-export declare const hasTestBlock: (text: string | undefined) => boolean;
+import { type ExamplePolicy } from './doc-system/example-policy.js';
+export declare const hasTestBlock: (text: string | undefined, policy?: ExamplePolicy) => boolean;
 export interface PageTestResults {
     passed: boolean;
     tests: TestResults['tests'];

package/dist/doc-browser.js

@@ -156,10 +156,12 @@ import { buildSlugMap, pathForSlug, filenameForPath, } from './doc-system/routin
 import { buildNavTree } from './doc-system/nav-tree.js';
 import { renderDocMarkdown } from './doc-system/render.js';
 import { pageTitle } from './doc-system/doc-title.js';
-import { unsettledExamples } from './doc-system/test-completion.js';
+import { unsettledExamples, tallyExamples, closePage, } from './doc-system/test-completion.js';
 import { LiveExample, testManager, pageTestCount, enableTests, disableTests, } from './live-example.js';
 import { highlightBlocks } from './doc-system/highlight.js';
 import { examplePolicy } from './live-example/insert-examples.js';
+import { collectCodeTokens } from './doc-system/code-fences.js';
+import { isLiveFence } from './doc-system/example-policy.js';
 import { tosiSidenav, TosiSidenav } from './side-nav.js';
 import { icons } from './icons.js';
 import { tosiLocalized } from './localize.js';
@@ -183,8 +185,17 @@ standalone run reporting "62 passed" against a build the full suite failed on.
 A fence is only a fence at the start of a line, which is the rule the doc extractor already
 applies to `/*#`. `[ \t]*` allows the indented fences that appear inside list items.
 */
-const TEST_FENCE = /^[ \t]*```test[ \t]*$/m;
-export const hasTestBlock = (text) => typeof text === 'string' && TEST_FENCE.test(text);
+/*
+…and a line-anchored regex was not the rule either (#2874). A ` ```test ` line SHOWN inside a
+longer or `:static` block matched it, so a page that documents the test tier was counted as a
+page with tests, ran in a background iframe, reported nothing, and was counted as tested. The
+lane said "19 of 19 pages" over results for 18. Code blocks now come from marked, the parser
+the page is rendered with, and liveness from `isLiveFence`, under the site's policy.
+*/
+export const hasTestBlock = (text, policy = 'auto') => typeof text === 'string' &&
+    // Cheap reject first: lexing every doc in a large corpus is not free.
+    text.includes('test') &&
+    collectCodeTokens(text).some((t) => t.lang === 'test' && isLiveFence('test', t.mode, policy));
 const { div, span, a, header, button, template, input, h2, img, details, summary, ul, li, } = elements;
 /*
 Test colors, and the text color that goes ON each — because contrast is a PAIR.
@@ -498,10 +509,15 @@ export function createDocBrowser(options) {
     let refreshNav = () => { };
     // Test result tracking
     const pageTestResults = {};
+    // The per-example results behind the entry for the page the reader is on.
+    let liveResults = new Map();
+    let liveResultsFile = '';
     let testResultsResolve;
     let backgroundTestsStarted = false;
     let pagesWithTests = 0;
     let pagesTested = 0;
+    // Per-page budget for the background runner; see the wait loop for why it exceeds 30s.
+    const PAGE_DEADLINE_MS = 45_000;
     // Set up global promise for scriptable browser integration. A memory-routed
     // (embedded) browser must not clobber the host page's global.
     if (!memoryRouting) {
@@ -554,15 +570,93 @@ export function createDocBrowser(options) {
     const handleTestComplete = (event) => {
         const { results } = event.detail;
         const filename = String(app.currentDoc.filename);
-        // Reset page results each time (don't accumulate across reloads)
-        pageTestResults[filename] = {
-            passed: results.failed === 0,
-            tests: [...results.tests],
-            totalPassed: results.passed,
-            totalFailed: results.failed,
-        };
+        /*
+        One entry per EXAMPLE, added up — `testcomplete` fires once for each. This used to
+        replace the page's entry on every event, so the last example to finish was the whole
+        page: a failing first example followed by a passing second one reported green.
+    
+        Keyed by element, so an example that re-runs (an edit, a refresh) replaces its own
+        result and nothing else's. A different page starts a fresh tally, and elements no longer
+        in the document are dropped first.
+        */
+        if (liveResultsFile !== filename) {
+            liveResultsFile = filename;
+            liveResults = new Map();
+        }
+        // Navigating away and back re-creates the page's examples; the old elements are gone
+        // from the document and their results must go with them, or the page counts double.
+        for (const element of liveResults.keys()) {
+            if (!element.isConnected)
+                liveResults.delete(element);
+        }
+        liveResults.set(event.detail.element, results);
+        pageTestResults[filename] = tallyExamples(liveResults.values());
         updateDocTestStatus(filename);
     };
+    /*
+    When has the page now rendered in THIS document finished its tests? Resolves with the
+    examples that never settled (empty when all did).
+  
+    One definition, used by a test iframe to signal its parent, by the page the reader is on,
+    and by a deployed page — the latter two used to be a flat timer, which marked a page tested
+    whether or not anything had reported.
+  
+    The census is only as good as the DOM it reads. Examples are inserted into the page as it
+    renders, so a census taken while insertion is still in progress sees FEWER examples than the
+    page has — and an example that appears afterwards was never waited on, never reported, and
+    never missed. So the count must be observed to STOP GROWING before "done" means anything:
+    two consecutive equal censuses, and a third example arriving later restarts the requirement.
+    */
+    const whenPageSettled = (deadlineMs) => new Promise((resolve) => {
+        const startedAt = Date.now();
+        let previousCount = -1;
+        let stableCensuses = 0;
+        const census = () => {
+            const examples = [...container.querySelectorAll('tosi-example')];
+            if (examples.length === previousCount)
+                stableCensuses++;
+            else
+                stableCensuses = 0;
+            previousCount = examples.length;
+            const stalled = unsettledExamples(examples.map((ex) => ({
+                element: ex,
+                test: ex.test,
+                hasTests: ex.classList.contains('-has-tests'),
+                testRunning: ex.classList.contains('-test-running'),
+            })));
+            const settling = stalled.length > 0 || stableCensuses < 2;
+            if (settling && Date.now() - startedAt < deadlineMs) {
+                setTimeout(census, 100);
+                return;
+            }
+            resolve(stalled.map(({ element }) => element));
+        };
+        // Give the page a moment to insert its examples before the first census.
+        setTimeout(census, 500);
+    });
+    /*
+    The page the reader is on runs its tests in place. Wait for them the way a test iframe is
+    waited for, then close the entry by the same rule (`closePage`): examples that never
+    settled, or no results at all, are failures with a name.
+    */
+    const settleCurrentPage = async (filename) => {
+        const stalled = await whenPageSettled(PAGE_DEADLINE_MS);
+        /*
+        If the reader navigated away, the census was of ANOTHER page and says nothing about this
+        one. Whatever this page had reported stands, and if it reported nothing no failure is
+        invented for it: the report then has one page fewer than it should, which is the honest
+        statement ("we did not look") and is what `testReportVerdict` fails on.
+        */
+        if (String(app.currentDoc.filename) === filename) {
+            pageTestResults[filename] = closePage(pageTestResults[filename], {
+                timedOut: false,
+                deadlineMs: PAGE_DEADLINE_MS,
+                stalled: stalled.length,
+            });
+            updateDocTestStatus(filename);
+        }
+        markPageTested(filename);
+    };
     // Track when a page finishes loading all its tests
     const markPageTested = (_filename) => {
         pagesTested++;
@@ -1769,36 +1863,7 @@ export function createDocBrowser(options) {
     if (isTestFrame && testFrameFilename) {
         // Generous: this bounds a hung example, it does not pace a slow one.
         const STALL_TIMEOUT_MS = 30000;
-        const startedAt = Date.now();
-        /*
-        The census is only as good as the DOM it reads. Examples are inserted into the page as
-        it renders, so a census taken while insertion is still in progress sees FEWER examples
-        than the page has — and an example that appears afterwards was never waited on, never
-        reported, and never missed. That is the same silent-loss failure as the bug above, one
-        step earlier, so the count must be observed to STOP GROWING before "done" means
-        anything. Two consecutive equal censuses is enough; a third example arriving later
-        restarts the requirement.
-        */
-        let previousCount = -1;
-        let stableCensuses = 0;
-        const signalDone = () => {
-            const examples = [...container.querySelectorAll('tosi-example')];
-            if (examples.length === previousCount)
-                stableCensuses++;
-            else
-                stableCensuses = 0;
-            previousCount = examples.length;
-            const stalled = unsettledExamples(examples.map((ex) => ({
-                element: ex,
-                test: ex.test,
-                hasTests: ex.classList.contains('-has-tests'),
-                testRunning: ex.classList.contains('-test-running'),
-            })));
-            const settling = stalled.length > 0 || stableCensuses < 2;
-            if (settling && Date.now() - startedAt < STALL_TIMEOUT_MS) {
-                setTimeout(signalDone, 100);
-                return;
-            }
+        void whenPageSettled(STALL_TIMEOUT_MS).then((stalled) => {
             /*
             Past the deadline with examples still unsettled: report them as failures rather than
             signalling a done that omits them. An example that never finishes is a real defect —
@@ -1812,7 +1877,7 @@ export function createDocBrowser(options) {
                     results: {
                         passed: 0,
                         failed: stalled.length,
-                        tests: stalled.map(({ element }) => ({
+                        tests: stalled.map((element) => ({
                             name: `example never finished running (${STALL_TIMEOUT_MS}ms)`,
                             passed: false,
                             error: `The example's code did not complete, so its test block never ran. Its source begins: ${String(element.js ?? '').slice(0, 120)}`,
@@ -1821,9 +1886,7 @@ export function createDocBrowser(options) {
                 }, '*');
             }
             window.parent.postMessage({ type: 'tosi-tests-done', filename: testFrameFilename }, '*');
-        };
-        // Give the page a moment to insert its examples before taking the census.
-        setTimeout(signalDone, 500);
+        });
     }
     // Background test runner for all doc pages
     const runBackgroundTests = async () => {
@@ -1847,7 +1910,7 @@ export function createDocBrowser(options) {
         exits 0. So a runtime error in an example on a page with no ` ```test ` block is caught by
         nothing. Tracked as its own issue rather than papered over by making this predicate loose.
         */
-        const docsWithTests = docs.filter((doc) => hasTestBlock(doc.text));
+        const docsWithTests = docs.filter((doc) => hasTestBlock(doc.text, examplePolicy()));
         pagesWithTests = docsWithTests.length;
         if (pagesWithTests > 0) {
             setTestWidgetRunning();
@@ -1895,6 +1958,9 @@ export function createDocBrowser(options) {
         const messageHandler = (event) => {
             if (event.data?.type !== 'tosi-test-results')
                 return;
+            // Only from OUR test frame: any other window on the page could otherwise post results.
+            if (event.source !== testFrame.contentWindow)
+                return;
             const { filename, results } = event.data;
             if (!pageTestResults[filename]) {
                 pageTestResults[filename] = {
@@ -1922,11 +1988,15 @@ export function createDocBrowser(options) {
                 routing === 'path'
                     ? `${window.location.origin}${hrefFor(doc.filename)}?_testMode=1`
                     : `${window.location.origin}${window.location.pathname}?${doc.filename}&_testMode=1`;
-            // Wait for the iframe to signal it's done (max 30s per page)
+            // Wait for the iframe to signal it's done. LONGER than the frame's own 30s stall
+            // timeout, so a frame that gives up on a hung example gets to say so: at the same 30s
+            // the two raced, and the parent could move on a moment before the stall report landed.
+            let timedOut = false;
             await new Promise((resolve) => {
-                const deadline = Date.now() + 30_000;
+                const deadline = Date.now() + PAGE_DEADLINE_MS;
                 const onDone = (event) => {
-                    if (event.data?.type === 'tosi-tests-done' &&
+                    if (event.source === testFrame.contentWindow &&
+                        event.data?.type === 'tosi-tests-done' &&
                         event.data.filename === doc.filename) {
                         window.removeEventListener('message', onDone);
                         resolve();
@@ -1935,17 +2005,24 @@ export function createDocBrowser(options) {
                 window.addEventListener('message', onDone);
                 setTimeout(() => {
                     window.removeEventListener('message', onDone);
+                    timedOut = true;
                     resolve();
                 }, deadline - Date.now());
             });
+            // A page that timed out fails even if some of its examples had reported (closePage).
+            pageTestResults[doc.filename] = closePage(pageTestResults[doc.filename], {
+                timedOut,
+                deadlineMs: PAGE_DEADLINE_MS,
+            });
+            updateDocTestStatus(doc.filename);
             markPageTested(doc.filename);
         }
         // Clean up
         window.removeEventListener('message', messageHandler);
         testFrame.remove();
-        // Mark current page as tested if it has tests
+        // The page the reader is on ran its tests in place; wait for them and close its entry.
         if (docsWithTests.some((d) => d.filename === currentFilename)) {
-            setTimeout(() => markPageTested(currentFilename), 1000);
+            void settleCurrentPage(currentFilename);
         }
     };
     // Run background tests when enabled (initially or when toggled on)
@@ -1956,11 +2033,20 @@ export function createDocBrowser(options) {
             setTimeout(runBackgroundTests, 1000);
         }
         else {
-            const currentHasTests = currentDoc.text.includes('```test');
+            /*
+            The page showing NOW, not the one this browser was constructed on. Tests are toggled
+            from the menu, often after navigating, and the construction-time doc then named a page
+            that is no longer on screen: it was judged by a census of some other page and recorded
+            as "reported no tests" without ever having run (1.16.8 re-review, F1).
+            */
+            const showing = String(app.currentDoc.filename);
+            const showingDoc = docs.find((d) => d.filename === showing);
+            const currentHasTests = hasTestBlock(showingDoc?.text, examplePolicy());
             if (currentHasTests) {
                 pagesWithTests = 1;
                 setTestWidgetRunning();
-                setTimeout(() => markPageTested(currentDoc.filename), 2000);
+                // Not a flat timer: the same settle-and-close rule as on localhost.
+                void settleCurrentPage(showing);
             }
             else if (testResultsResolve) {
                 testResultsResolve({

package/dist/doc-system/{site/code-fences.js → code-fences.js}

@@ -1,13 +1,14 @@
 /*
-The build's ONE way to enumerate the code blocks in a doc: marked's own lexer.
+THE way to enumerate the code blocks in a doc, build and client alike: marked's own lexer.
 
 `check-examples` had this function; `build-warnings` then grew a hand-written line scanner for
 the same job, and the 1.16.7 re-review found the scanner disagreeing with marked on CRLF files,
 blockquoted fences and fences inside HTML blocks. A scanner can only ever approximate the
-parser the pages are rendered with, so nothing here scans: both callers ask marked.
+parser the pages are rendered with, so nothing here scans: every caller asks marked. The
+doc-browser's test runner asks here too (`hasTestBlock`), for the same reason.
 */
 import { marked } from 'marked';
-import { parseFenceInfo } from '../example-policy.js';
+import { parseFenceInfo } from './example-policy.js';
 /*
 Fence info is parsed by `parseFenceInfo`, the one copy of that grammar. This file had its own
 `dialectOf`/`modeOf` pair; once fences could carry JSON options (#184) `modeOf` would have read

package/dist/doc-system/site/build-warnings.js

@@ -8,7 +8,7 @@ each of these is a legitimate thing to do on purpose. `strict: true` in the site
 project that never does them on purpose says so, and then the build fails.
 */
 import { isLiveFence } from '../example-policy.js';
-import { collectCodeTokens } from './code-fences.js';
+import { collectCodeTokens } from '../code-fences.js';
 /** Thrown by `misconfigured` under `strict`, so a caller can tell it from a crash. */
 export class SiteMisconfiguredError extends Error {
     name = 'SiteMisconfiguredError';

package/dist/doc-system/site/check-examples.js

@@ -21,7 +21,7 @@ executed, so they're skipped.
 
 Build-time only (bun). Never import from browser code.
 */
-import { collectCodeTokens } from './code-fences.js';
+import { collectCodeTokens } from '../code-fences.js';
 import { rewriteImports, AsyncFunction, loadTransform, transformAvailable, UnsupportedImportError, } from '../../live-example/code-transform.js';
 import { isLiveFence } from '../example-policy.js';
 // The default live-example context (matches the IIFE globals the pages provide).

package/dist/doc-system/site/dev-server.js

@@ -7,6 +7,7 @@ headless browser through the inline doc tests and exits with their pass/fail.
 
 Build-time only (Bun APIs). Never import this from browser code.
 */
+import { testReportVerdict } from './test-report.js';
 import * as path from 'path';
 import { editableSourcePaths, mayEditSource } from './editable-sources.js';
 import { statSync, existsSync, readFileSync } from 'fs';
@@ -1128,8 +1129,12 @@ export async function devServer(config, opts = {}) {
                 }
                 console.error('');
             }
+            else if (results.passed > 0 && !testReportVerdict(results).ok) {
+                // Also during a plain `bun start`: an incomplete run must not be silent (#2874).
+                console.error(`\n❌ Browser tests: ${testReportVerdict(results).reasons.join('; ')}\n`);
+            }
             else if (results.passed > 0) {
-                console.log(`\n✅ Browser tests: ${results.passed} passed\n`);
+                console.log(`\n✅ Browser tests: ${results.passed} passed across ${Object.keys(results.pages ?? {}).length} pages\n`);
             }
             if (testReportResolve && (results.passed > 0 || results.failed > 0)) {
                 testReportResolve(results);
@@ -2470,7 +2475,12 @@ export async function devServer(config, opts = {}) {
         }
         try {
             const results = await testResults;
-            const exitCode = results.failed > 0 ? 1 : 0;
+            // The verdict covers the pages that did NOT report, not just the tests that did (#2874).
+            const verdict = testReportVerdict(results);
+            if (!verdict.ok) {
+                console.error(`\n❌ Browser tests: ${verdict.reasons.join('; ')}\n`);
+            }
+            const exitCode = verdict.ok ? 0 : 1;
             await stopHaltija();
             shutdown(exitCode);
         }

package/dist/doc-system/site/index.d.ts

@@ -2,6 +2,8 @@ export { defineSiteConfig } from './site-config.js';
 export type { SiteConfig, SiteHost, LibraryBuildContext, } from './site-config.js';
 export { buildSite } from './orchestrator.js';
 export { SiteMisconfiguredError } from './build-warnings.js';
+export { testReportVerdict } from './test-report.js';
+export type { TestReportLike, TestReportVerdict } from './test-report.js';
 export { devServer } from './dev-server.js';
 export { auditDependencies, reportAudit, resolveAuditMode, } from './audit-guard.js';
 export type { AuditConfig, AuditGate, AuditResult, AuditSeverity, AuditMode, AuditAdvisory, } from './audit-guard.js';

package/dist/doc-system/site/index.js

@@ -8,6 +8,9 @@ export { buildSite } from './orchestrator.js';
 // Thrown by `buildSite` under `strict: true` — exported so a caller can tell a refused
 // misconfiguration from a crash (`catch (e) { if (e instanceof SiteMisconfiguredError) … }`).
 export { SiteMisconfiguredError } from './build-warnings.js';
+// THE pass/fail rule for a browser-test report (`window.__docTestResults`), for adopters'
+// own gates: it fails on pages that never reported, not only on failed tests.
+export { testReportVerdict } from './test-report.js';
 export { devServer } from './dev-server.js';
 export { auditDependencies, reportAudit, resolveAuditMode, } from './audit-guard.js';
 export { openDevBrowser, buildOpenPlan } from './open-browser.js';

package/dist/doc-system/site/test-report.d.ts

@@ -0,0 +1,13 @@
+export interface TestReportLike {
+    passed?: number;
+    failed?: number;
+    pages?: Record<string, unknown>;
+    pagesWithTests?: number;
+    pagesTested?: number;
+}
+export interface TestReportVerdict {
+    ok: boolean;
+    /** One sentence per reason the report is not a pass; empty when `ok`. */
+    reasons: string[];
+}
+export declare function testReportVerdict(report: TestReportLike): TestReportVerdict;

package/dist/doc-system/site/test-report.js

@@ -0,0 +1,33 @@
+/*
+Whether a browser-test report is a PASS — decided here, by one function, for every lane.
+
+The lane used to pass on `failed === 0`. That is the verdict for the tests that reported; it
+says nothing about the ones that did not, and a run once passed with results for 7 of 19
+pages (board #2874). The report already carried the numbers to notice — nobody compared them.
+*/
+export function testReportVerdict(report) {
+    const reasons = [];
+    const failed = report.failed ?? 0;
+    const passed = report.passed ?? 0;
+    const expected = report.pagesWithTests ?? 0;
+    const tested = report.pagesTested ?? 0;
+    const recorded = Object.keys(report.pages ?? {}).length;
+    if (failed > 0)
+        reasons.push(`${failed} test(s) failed`);
+    if (passed + failed === 0)
+        reasons.push('no tests ran');
+    // A report that ran tests but cannot say over how many pages is not evidence of coverage.
+    // (Missing fields read as 0, so without this `{ passed: 5 }` alone would be a pass.)
+    if (passed + failed > 0 && expected === 0) {
+        reasons.push('the report does not say how many pages have tests');
+    }
+    if (tested < expected) {
+        reasons.push(`only ${tested} of ${expected} pages with tests were run`);
+    }
+    // FEWER, not "different": a page with no test blocks whose example threw also records an
+    // entry (that failure is counted above), and must not read as "20 of 19".
+    if (recorded < expected) {
+        reasons.push(`results were recorded for ${recorded} of ${expected} pages with tests`);
+    }
+    return { ok: reasons.length === 0, reasons };
+}

package/dist/doc-system/test-completion.d.ts

@@ -19,3 +19,46 @@ export declare function isSettled(example: ExampleTestState): boolean;
  * complete while that example had yet to run a single assertion.
  */
 export declare function unsettledExamples<T extends ExampleTestState>(examples: T[]): T[];
+interface ExampleResults {
+    passed: number;
+    failed: number;
+    tests: Array<{
+        name: string;
+        passed: boolean;
+        error?: string;
+    }>;
+}
+/** A page's results, as the doc-browser records and reports them (`PageTestResults`). */
+export interface PageTally {
+    passed: boolean;
+    tests: ExampleResults['tests'];
+    totalPassed: number;
+    totalFailed: number;
+}
+/**
+ * Add up a page from its examples' results, one entry per example.
+ *
+ * Per EXAMPLE, because `testcomplete` fires once for each. The page the reader is on used to
+ * REPLACE its entry on every event, so the last example to finish was the whole page: a
+ * failing first example followed by a passing second one reported green.
+ */
+export declare function tallyExamples(results: Iterable<ExampleResults>): PageTally;
+/** How waiting for a page ended. */
+export interface PageOutcome {
+    /** the deadline passed before the page said it was done */
+    timedOut: boolean;
+    /** the deadline, for the message */
+    deadlineMs: number;
+    /** examples with tests that never settled (see `unsettledExamples`) */
+    stalled?: number;
+}
+/**
+ * Close a page's entry once waiting for it is over. Returns what to record.
+ *
+ * A page that timed out FAILS, whether or not some of its examples had reported: results for
+ * one example say nothing about the ones still hung behind it, and "0 failed so far" was
+ * being read as a pass. A page that finished with no results at all fails too — it has test
+ * blocks, and none of them ran.
+ */
+export declare function closePage(existing: PageTally | undefined, outcome: PageOutcome): PageTally;
+export {};

package/dist/doc-system/test-completion.js

@@ -25,3 +25,57 @@ export function isSettled(example) {
 export function unsettledExamples(examples) {
     return examples.filter((example) => example.test && !isSettled(example));
 }
+/**
+ * Add up a page from its examples' results, one entry per example.
+ *
+ * Per EXAMPLE, because `testcomplete` fires once for each. The page the reader is on used to
+ * REPLACE its entry on every event, so the last example to finish was the whole page: a
+ * failing first example followed by a passing second one reported green.
+ */
+export function tallyExamples(results) {
+    const tally = {
+        passed: true,
+        tests: [],
+        totalPassed: 0,
+        totalFailed: 0,
+    };
+    for (const r of results) {
+        tally.tests.push(...r.tests);
+        tally.totalPassed += r.passed;
+        tally.totalFailed += r.failed;
+    }
+    tally.passed = tally.totalFailed === 0;
+    return tally;
+}
+/**
+ * Close a page's entry once waiting for it is over. Returns what to record.
+ *
+ * A page that timed out FAILS, whether or not some of its examples had reported: results for
+ * one example say nothing about the ones still hung behind it, and "0 failed so far" was
+ * being read as a pass. A page that finished with no results at all fails too — it has test
+ * blocks, and none of them ran.
+ */
+export function closePage(existing, outcome) {
+    const seconds = outcome.deadlineMs / 1000;
+    const fail = (name, error) => {
+        const base = existing ?? tallyExamples([]);
+        return {
+            passed: false,
+            tests: [...base.tests, { name, passed: false, error }],
+            totalPassed: base.totalPassed,
+            totalFailed: base.totalFailed + 1,
+        };
+    };
+    if (outcome.stalled) {
+        return fail('every example with tests finished', `${outcome.stalled} example(s) with test blocks had not finished after ${seconds}s, so their tests did not run.`);
+    }
+    if (outcome.timedOut) {
+        return fail('page finished its tests in time', existing
+            ? `The page did not finish within ${seconds}s. Some tests reported; the rest did not run.`
+            : `The page did not finish within ${seconds}s and reported no tests, so its test blocks did not run.`);
+    }
+    if (!existing) {
+        return fail('page reported test results', 'The page finished but reported no tests, though it has test blocks.');
+    }
+    return existing;
+}