tosijs-ui 1.17.0-beta.2 → 1.17.0-beta.4

package/CHANGELOG.md

@@ -1,6 +1,41 @@
 # Changelog
 
-## 1.17.0-beta.2
+## 1.17.0-beta.4
+
+> Prerelease, published under the `beta` dist-tag: install with `tosijs-ui@beta`. `latest` stays
+> on 1.16.41. Fixes from Learn 3D and from tosijs-native's first site build.
+
+- **Every doc page can be scrolled by keyboard, and has a main landmark (#3575).** A page of
+  plain prose had nothing to tab to inside its scrolling content, so a keyboard user could not
+  scroll it, which axe reports as `scrollable-region-focusable` (Learn 3D: 71 of 126 pages). The
+  content scroller is now focusable, and it is the page's main landmark, which the doc site did not
+  have at all: a screen reader's "jump to main content" now works. A doc browser embedded in
+  another page stays focusable without claiming that page's main landmark.
+- **`buildSite` says it needs `happy-dom` before it starts (#3600).** Without it, the build died
+  partway through with "Cannot find package 'happy-dom'" from a file inside tosijs-ui. It now
+  stops before writing anything and prints the install command. `happy-dom` stays an optional
+  peer: a project that only uses the components never needs it.
+- **TJS's own doc comments, `/# … #/`, become documentation (#3601).** In `.tjs` and `.ajs`
+  files they are read in source order with any `/*#` blocks. tjs-lang finds them, so its grammar
+  applies exactly; that needs a tjs-lang that exports `findDocCommentSpans` (requested). With an
+  older one, the build names each file whose `/#` comments it could not read, instead of
+  dropping the page silently. `.ajs` files are now scraped for `/*#` blocks too.
+
+### 1.17.0-beta.3 (prerelease detail)
+
+> Prerelease, published under the `beta` dist-tag: install with `tosijs-ui@beta`. `latest` stays
+> on 1.16.41. beta.2 was staged and never published; its notes follow.
+
+- **The axe audit waits for a page marked `aria-busy`, and one slow page no longer ends it.** A
+  page that is still loading (a 3D scene fetching large models, say) marks an element
+  `aria-busy="true"`, the standard ARIA signal, and is waited for up to 30 seconds; other pages
+  are audited once they have stopped adding elements, or after five seconds. A page that never
+  loads, or is still busy at 30 seconds, is listed as not fully checked, and the build makes no
+  claim about it. Before, each page load waited for the network to go idle, so a page that kept
+  downloading for 30 seconds, or that polled, threw and ended the audit of every page. The doc
+  site reference shows a component setting and clearing `aria-busy`.
+
+### 1.17.0-beta.2 (prerelease detail)
 
 > Prerelease, published under the `beta` dist-tag: install with `tosijs-ui@beta`. `latest` stays
 > on 1.16.41. Fixes from Learn 3D's first run of beta.1.

package/dist/doc-browser.js

@@ -1699,7 +1699,17 @@ export function createDocBrowser(options) {
             // The sidenav has just introduced itself; give it the layout the doc asked for.
             applyFullScreen();
         },
-    }, navContent, div({
+    }, navContent, 
+    /*
+    The page's main landmark, and focusable: a page of plain prose has nothing else to tab
+    to, so a keyboard user could not scroll it at all (WCAG 2.1.1; axe
+    scrollable-region-focusable, #3575). A role, not a <main>, so styles written against the
+    div still apply. An embedded browser (`memory` routing) is part of someone else's page,
+    so it does not claim that page's main landmark.
+    */
+    div({
+        ...(routing === 'memory' ? {} : { role: 'main' }),
+        tabindex: 0,
         style: {
             position: 'relative',
             overflowY: 'scroll',

package/dist/doc-system/site/a11y-audit-cli.js

@@ -10,7 +10,7 @@ tosijs-ui. When either is missing, or Chromium is not installed, it says which a
 NOT_RUN_EXIT, so the build can say the audit did not run instead of reporting a pass.
 
   bun a11y-audit-cli.ts <payload.json>   # payload = { outputDir, basePath, pages }
-  stdout: one JSON line, { violations: AxeFinding[] }
+  stdout: one JSON line, { violations: AxeFinding[], unsettled: { page, reason }[] }
 
 Build-time only. Never import this from browser code.
 */
@@ -19,7 +19,7 @@ import { createRequire } from 'module';
 import { existsSync } from 'fs';
 import { NOT_RUN_EXIT, AXE_TAGS } from './a11y-audit.js';
 const payloadPath = process.argv[2];
-const { outputDir, basePath, pages } = JSON.parse(await Bun.file(payloadPath).text());
+const { outputDir, basePath, pages, slowPageMs } = JSON.parse(await Bun.file(payloadPath).text());
 const notRun = (why) => {
     console.error(why);
     process.exit(NOT_RUN_EXIT);
@@ -69,7 +69,12 @@ Pages are audited a few at a time, each in its own tab: one at a time took Learn
 to ~73s (#3585). Results are kept in page order, so the report reads the same either way.
 */
 const CONCURRENCY = 4;
+/** How long a page may take to fire `load`, and to finish being `aria-busy`. */
+const SLOW_PAGE_MS = slowPageMs ?? 30_000;
+/** How long an un-busy page may keep adding elements before it is audited as it stands. */
+const QUIET_CAP_MS = 5_000;
 const perPage = [];
+const unsettled = [];
 try {
     const context = await browser.newContext();
     /*
@@ -83,9 +88,32 @@ try {
         while (next < pages.length) {
             const index = next++;
             const pagePath = pages[index];
-            await page.goto(`http://localhost:${server.port}${prefix}${pagePath}`, {
-                waitUntil: 'networkidle',
-            });
+            /*
+            One slow or broken page must not end the audit of every other page, and must not pass
+            unremarked either: it is listed as not fully checked.
+            */
+            try {
+                await page.goto(`http://localhost:${server.port}${prefix}${pagePath}`, {
+                    waitUntil: 'load',
+                    timeout: SLOW_PAGE_MS,
+                });
+            }
+            catch (error) {
+                const message = error.message.split('\n')[0];
+                if (!/timeout/i.test(message)) {
+                    unsettled.push({ page: pagePath, reason: `not audited: ${message}` });
+                    perPage[index] = [];
+                    continue;
+                }
+                unsettled.push({
+                    page: pagePath,
+                    reason: `did not finish loading in ${SLOW_PAGE_MS / 1000}s; audited as it stood`,
+                });
+            }
+            // Best effort: a page that polls or streams never goes idle, and that is not a defect.
+            await page
+                .waitForLoadState('networkidle', { timeout: 10_000 })
+                .catch(() => undefined);
             // Audit the page a reader uses: hydrated when the bundle loads, static if it does not.
             if (await page.$('tosi-doc-system'))
                 await page
@@ -94,26 +122,42 @@ try {
                 })
                     .catch(() => undefined);
             /*
-            Then until no element has been added or removed for half a second (at most five): live examples render
-            after hydration, and a fixed pause audited a busy machine's pages half-built, so the
-            findings changed from run to run.
+            Then until the page has settled: no element added or removed for half a second, and
+            nothing marked `aria-busy="true"`. A fixed pause audited a busy machine's pages
+            half-built. A page that is still loading (a 3D scene fetching its models, say) says so with
+            aria-busy, which screen readers also announce, and is waited for up to SLOW_PAGE_MS; one
+            that merely keeps changing is audited after QUIET_CAP_MS.
             */
-            await page.evaluate(() => new Promise((resolve) => {
-                let timer = setTimeout(done, 500);
-                const deadline = setTimeout(done, 5000);
+            const settled = await page.evaluate(({ quietCap, busyCap }) => new Promise((resolve) => {
+                const start = performance.now();
+                let last = start;
                 const observer = new MutationObserver(() => {
-                    clearTimeout(timer);
-                    timer = setTimeout(done, 500);
+                    last = performance.now();
                 });
-                function done() {
-                    observer.disconnect();
-                    clearTimeout(timer);
-                    clearTimeout(deadline);
-                    resolve();
-                }
                 // Elements arriving, not attributes: an animation changes a style forever.
                 observer.observe(document, { subtree: true, childList: true });
-            }));
+                const tick = () => {
+                    const now = performance.now();
+                    const busy = !!document.querySelector('[aria-busy="true"]');
+                    const outcome = now - last >= 500 && !busy
+                        ? 'settled'
+                        : !busy && now - start >= quietCap
+                            ? 'changing'
+                            : now - start >= busyCap
+                                ? 'busy'
+                                : '';
+                    if (!outcome)
+                        return void setTimeout(tick, 100);
+                    observer.disconnect();
+                    resolve(outcome);
+                };
+                tick();
+            }), { quietCap: QUIET_CAP_MS, busyCap: SLOW_PAGE_MS });
+            if (settled === 'busy')
+                unsettled.push({
+                    page: pagePath,
+                    reason: `still aria-busy after ${SLOW_PAGE_MS / 1000}s; audited as it stood`,
+                });
             await page.addScriptTag({ path: axePath });
             const found = await page.evaluate(async (tags) => {
                 const result = await window.axe.run(document, {
@@ -136,4 +180,4 @@ finally {
     server.stop(true);
 }
 const violations = perPage.flat();
-console.log(JSON.stringify({ violations }));
+console.log(JSON.stringify({ violations, unsettled }));

package/dist/doc-system/site/a11y-audit.d.ts

@@ -11,9 +11,15 @@ export interface AxeFinding {
     /** a CSS selector for each element that fails the rule */
     targets: string[];
 }
+/** A page the audit could not check fully: it never loaded, or never stopped being busy. */
+export interface UnsettledPage {
+    page: string;
+    reason: string;
+}
 export type AxeAuditResult = {
     ran: true;
     violations: AxeFinding[];
+    unsettled: UnsettledPage[];
 } | {
     ran: false;
     reason: string;
@@ -25,6 +31,8 @@ export declare function auditBuiltSite(opts: {
     pages: string[];
     /** the project Playwright and axe-core are resolved from (default: the working directory) */
     projectDir?: string;
+    /** how long a page may take to load, or stay `aria-busy="true"` (default 30s) */
+    slowPageMs?: number;
 }): Promise<AxeAuditResult>;
 /** The report: one line per rule per page, worst impact first. */
 export declare function formatAxeFindings(violations: AxeFinding[]): string;

package/dist/doc-system/site/a11y-audit.js

@@ -25,6 +25,7 @@ export async function auditBuiltSite(opts) {
         outputDir: path.resolve(opts.outputDir),
         basePath: opts.basePath ?? '',
         pages: opts.pages,
+        slowPageMs: opts.slowPageMs,
     }));
     // `--no-install`: without it Bun resolves a package the project does not have from its global
     // cache, so "not installed" would never be said and the audit would use a Playwright nobody chose.
@@ -42,8 +43,10 @@ export async function auditBuiltSite(opts) {
         return { ran: false, reason: stderr.trim() };
     const line = stdout.trim().split('\n').pop() ?? '';
     try {
-        if (code === 0)
-            return { ran: true, ...JSON.parse(line) };
+        if (code === 0) {
+            const { violations, unsettled = [] } = JSON.parse(line);
+            return { ran: true, violations, unsettled };
+        }
     }
     catch {
         // fall through: a run that printed no result did not run

package/dist/doc-system/site/build-peers.d.ts

@@ -0,0 +1,6 @@
+/**
+ * The install command for each build peer that cannot be resolved, or none.
+ * `resolve` is injectable for tests.
+ */
+export declare function missingBuildPeers(moduleDir: string, resolve?: (name: string, from: string) => string): string[];
+export declare function missingBuildPeersMessage(missing: string[]): string;

package/dist/doc-system/site/build-peers.js

@@ -0,0 +1,43 @@
+/*
+Optional peers a site BUILD cannot do without (#3600).
+
+`happy-dom` is an optional peer of tosijs-ui because a project that only uses the components
+never needs it, and npm installs a required peer whether or not it is used. But `buildSite`
+does need it: the theme stylesheet is generated with tosijs imported under a DOM shim. Without
+it the build died several steps in with "Cannot find package 'happy-dom'" from a file inside
+tosijs-ui, which says nothing about whose job it is to install it. So the build checks first,
+and says what to run.
+*/
+import { readFileSync } from 'fs';
+import * as path from 'path';
+import { tosijsUiDir } from './codemirror-copies.js';
+/** What a site build requires, resolved from tosijs-ui's own location (where it is imported). */
+const BUILD_PEERS = ['happy-dom'];
+/**
+ * The install command for each build peer that cannot be resolved, or none.
+ * `resolve` is injectable for tests.
+ */
+export function missingBuildPeers(moduleDir, resolve = Bun.resolveSync) {
+    let ranges = {};
+    try {
+        const pkg = JSON.parse(readFileSync(path.join(tosijsUiDir(moduleDir), 'package.json'), 'utf8'));
+        ranges = pkg.peerDependencies ?? {};
+    }
+    catch {
+        // no manifest found: name the package without a range
+    }
+    return BUILD_PEERS.filter((name) => {
+        try {
+            resolve(name, moduleDir);
+            return false;
+        }
+        catch {
+            return true;
+        }
+    }).map((name) => (ranges[name] ? `${name}@${ranges[name]}` : name));
+}
+export function missingBuildPeersMessage(missing) {
+    return (`\n🛑 Building a site needs ${missing.join(', ')}, an optional peer of tosijs-ui ` +
+        `(a project that only uses the components does not).\n` +
+        `   bun add -d ${missing.map((m) => `'${m}'`).join(' ')}\n`);
+}

package/dist/doc-system/site/docs.d.ts

@@ -143,5 +143,9 @@ export declare function parseFrontmatter(content: string): {
     body: string;
 };
 export declare const SCRAPED_SOURCE_EXTENSIONS: string[];
+type DocSpans = (source: string) => Array<[number, number]>;
+/** Tests only: stand in for tjs-lang's scanner (`undefined` restores the lookup). */
+export declare function setTjsDocCommentScanner(scan: DocSpans | null | undefined): void;
 export declare function extractDocs(options: ExtractDocsOptions): Doc[];
 export declare function saveDocsJSON(docs: Doc[], outputPath: string): void;
+export {};

package/dist/doc-system/site/docs.js

@@ -30,8 +30,13 @@ whatever metadata the doc declares (`pin`, `order`, `parent`, `layout`, `descrip
 ## What is collected
 
 - **`.md` files**, whole. YAML frontmatter is read and removed.
-- **`/*#` … `*‎/` blocks** in `.ts`, `.tjs`, `.js` and `.css` files, as markdown. The first heading
-  is the page title. A block counts only when `/*#` starts a line.
+- **`/*#` … `*‎/` blocks** in `.ts`, `.tjs`, `.ajs`, `.js` and `.css` files, as markdown. The
+  first heading is the page title. A block counts only when `/*#` starts a line.
+- **TJS doc comments, `/#` … `#/`**, in `.tjs` and `.ajs` files, in source order with any `/*#`
+  blocks. They are found by tjs-lang itself, so they follow its grammar exactly: multi-line, and
+  a doc comment nested inside one is quoted as part of it. This needs a tjs-lang that exports
+  `findDocCommentSpans`; with an older one, the build names each file whose `/#` comments it
+  could not read.
 - **Files whose name starts with `_`** are skipped: use the prefix for drafts and fragments.
 
 Metadata blocks (`<!--{ … }-->` in markdown, `/*{ … }*‎/` in code), nav order, sections and
@@ -40,6 +45,7 @@ which code fences run live are described under [Doc format](/doc-site-system/#do
 /*{ "parent": "Documentation System", "order": 60, "description": "extractDocs() collects markdown files and doc comments from source code into one docs.json corpus, the input to the doc browser and static site." }*/
 // TODO CLI options
 import * as fs from 'fs';
+import { createRequire } from 'module';
 import * as path from 'path';
 import { truncationWarnings, formatTruncationWarnings, } from './truncated-doc.js';
 import { pinnedSort } from '../nav-tree.js';
@@ -230,9 +236,47 @@ ends it — the same trap tosijs-ui#70 records for doc blocks, hit again while w
 
 See `warnUnscrapedDocBlocks` below: a missing extension should never again be invisible.
 */
-export const SCRAPED_SOURCE_EXTENSIONS = ['.ts', '.js', '.tjs', '.css'];
+export const SCRAPED_SOURCE_EXTENSIONS = ['.ts', '.js', '.tjs', '.ajs', '.css'];
 /** Files that look documented but are not scraped — reported once at the end of a walk. */
 const unscrapedDocBlocks = [];
+/*
+TJS's own doc comment, `/# … #/` (#3601): multi-line, nestable, and the convention in `.tjs` and
+`.ajs` sources. Its grammar belongs to tjs-lang, which finds the spans over a literal-masked view
+(a raw scan reads a regex `/#/` and a later string `"#/"` as one comment). So this does not keep a
+second copy: it asks the installed tjs-lang, and when that tjs-lang does not export the scanner,
+the build names the files whose doc comments it could not read rather than dropping them.
+*/
+const TJS_DOC_EXTENSIONS = ['.tjs', '.ajs'];
+// An opener at the start of a line that its own line does not close: a single-line `/# x #/` is
+// a regex, never a doc comment.
+const TJS_DOC_OPENER = /^[ \t]*\/#(?![^\n]*#\/)/m;
+let tjsDocSpans;
+function tjsDocCommentSpans() {
+    if (tjsDocSpans !== undefined)
+        return tjsDocSpans;
+    tjsDocSpans = null;
+    // The project's tjs-lang, the one its .tjs files are written for.
+    const projectModule = createRequire(path.resolve('package.json'));
+    for (const entry of ['tjs-lang/lang', 'tjs-lang']) {
+        try {
+            const found = projectModule(entry).findDocCommentSpans;
+            if (typeof found === 'function') {
+                tjsDocSpans = found;
+                break;
+            }
+        }
+        catch {
+            // not installed, or no such entry: try the next
+        }
+    }
+    return tjsDocSpans;
+}
+/** Tests only: stand in for tjs-lang's scanner (`undefined` restores the lookup). */
+export function setTjsDocCommentScanner(scan) {
+    tjsDocSpans = scan;
+}
+/** `.tjs`/`.ajs` files with doc comments the installed tjs-lang could not read. */
+const unreadTjsDocComments = [];
 /**
  * Does an UNSCRAPED file appear to carry a doc block?
  *
@@ -385,9 +429,41 @@ walked) {
                 // before the slash). This keeps a `/*#` that appears inside a // comment,
                 // a string, or otherwise mid-line from being scraped as a spurious doc.
                 // m[1] is the block without its leading indentation.
-                const docs = [...content.matchAll(/^[ \t]*(\/\*#[\s\S]+?\*\/)/gm)].map((m) => m[1]);
+                const blocks = [
+                    ...content.matchAll(/^[ \t]*(\/\*#[\s\S]+?\*\/)/gm),
+                ].map((m) => ({
+                    at: m.index + m[0].length - m[1].length,
+                    block: m[1],
+                }));
+                const docs = blocks.map(({ block }) => block.substring(3, block.length - 2).trim());
+                // TJS doc comments, in source order with any `/*#` blocks.
+                if (TJS_DOC_EXTENSIONS.includes(path.extname(file)) &&
+                    TJS_DOC_OPENER.test(content)) {
+                    const scan = tjsDocCommentSpans();
+                    if (!scan) {
+                        unreadTjsDocComments.push(path.relative(process.cwd(), filePath));
+                    }
+                    else {
+                        const spans = scan(content);
+                        // Outermost only: a nested doc comment is being quoted, not published.
+                        const outer = spans.filter(([a, b]) => !spans.some(([c, d]) => c < a && b <= d));
+                        const all = [
+                            ...blocks.map(({ at }, i) => ({ at, text: docs[i] })),
+                            ...outer.map(([a, b]) => ({
+                                at: a,
+                                // The span may or may not include its delimiters; the text never does.
+                                text: content
+                                    .slice(a, b)
+                                    .replace(/^\/#/, '')
+                                    .replace(/#\/$/, '')
+                                    .trim(),
+                            })),
+                        ].sort((x, y) => x.at - y.at);
+                        docs.splice(0, docs.length, ...all.map((d) => d.text));
+                    }
+                }
                 if (docs.length) {
-                    const markdown = docs.map((s) => s.substring(3, s.length - 2).trim());
+                    const markdown = docs;
                     // A doc block that ended early drops the rest of the documentation silently, and
                     // turns the code after it back into source — which surfaces as parse errors
                     // pointing at prose rather than at the delimiter responsible (#70).
@@ -444,6 +520,14 @@ walked) {
     can act on without going looking. The fix is nearly always one entry in
     SCRAPED_SOURCE_EXTENSIONS.
     */
+    if (unreadTjsDocComments.length) {
+        console.warn(`\n⚠️  ${unreadTjsDocComments.length} file(s) have TJS doc comments (/# … #/) that this ` +
+            `build cannot read, so they have no page from them:\n` +
+            unreadTjsDocComments.map((f) => `    ${f}`).join('\n') +
+            `\n    Reading them needs findDocCommentSpans from tjs-lang, which the installed ` +
+            `tjs-lang does not export. Until it does, write those blocks as /*# … */.\n`);
+        unreadTjsDocComments.length = 0;
+    }
     if (unscrapedDocBlocks.length) {
         const list = unscrapedDocBlocks.map((f) => `    ${f}`).join('\n');
         console.warn(`\n⚠️  ${unscrapedDocBlocks.length} file(s) look documented but are not scraped, so ` +

package/dist/doc-system/site/orchestrator.js

@@ -31,6 +31,7 @@ export function registerSiteDialects(config) {
 import { namedBooks, partitionByBook, DEFAULT_BOOK } from '../book-target.js';
 import { EPUB_IMAGE_EXIT, attachCovers, listEpubVolumes, printCoverManifestUrl, VOLUME_MANIFEST, renderEpubDownloads, } from './epub-volumes.js';
 import { buildSlugMap, pathForSlug } from '../routing.js';
+import { missingBuildPeers, missingBuildPeersMessage } from './build-peers.js';
 import { assetStampInputs, computeAssetStamp, missingStampInputWarning, } from './asset-stamp.js';
 import { existsSync, mkdirSync, readdirSync, readFileSync, unlinkSync, } from 'fs';
 import { tmpdir } from 'os';
@@ -416,6 +417,12 @@ export async function buildSite(config, opts = {}) {
     }))) {
         return false;
     }
+    // A peer the build needs, checked before anything is deleted or written (#3600).
+    const missingPeers = missingBuildPeers(import.meta.dir);
+    if (missingPeers.length) {
+        console.error(missingBuildPeersMessage(missingPeers));
+        return false;
+    }
     // Dependency-audit gate. Runs on the initial build only — the dev server runs its
     // own synchronous audit just before binding the port, and `opts.skipAudit` keeps
     // watch rebuilds off the network. Fails the build (before the destructive rm -rf)
@@ -1706,12 +1713,25 @@ export async function buildSite(config, opts = {}) {
                     console.warn(`\n⚠️  Accessibility: the axe audit did not run, so this build makes no claim\n` +
                         `    about what it would find. ${audit.reason}\n`);
                 }
-                else if (audit.violations.length) {
-                    misconfigured(config.strict, `\n⚠️  Accessibility: axe found ${audit.violations.length} WCAG 2.2 AA failure(s) in the built pages:\n` +
-                        `${formatAxeFindings(audit.violations)}\n`);
-                }
                 else {
-                    console.log(`♿ axe: no WCAG 2.2 AA failures in ${pages.length} page(s)`);
+                    // A page the audit could not check fully is said, never counted as clean.
+                    if (audit.unsettled.length)
+                        console.warn(`\n⚠️  Accessibility: ${audit.unsettled.length} page(s) were not fully checked, so ` +
+                            `this build makes no claim about them. A page that is still loading should mark ` +
+                            `itself aria-busy="true"; the audit waits up to 30s for it:\n` +
+                            audit.unsettled
+                                .map((u) => `    ${u.page}  ${u.reason}`)
+                                .join('\n') +
+                            '\n');
+                    const checked = pages.length - audit.unsettled.length;
+                    if (audit.violations.length)
+                        misconfigured(config.strict, `\n⚠️  Accessibility: axe found ${audit.violations.length} WCAG 2.2 AA failure(s) in the built pages:\n` +
+                            `${formatAxeFindings(audit.violations)}\n`);
+                    else
+                        console.log(`♿ axe: no WCAG 2.2 AA failures in ${checked} fully checked page(s)` +
+                            (audit.unsettled.length
+                                ? ` (${audit.unsettled.length} not fully checked, above)`
+                                : ''));
                 }
             }
             console.timeEnd('build');