tosijs-ui 1.16.8 → 1.16.9

package/CHANGELOG.md

@@ -1,5 +1,73 @@
 # Changelog
 
+## 1.16.9
+
+Two sets of asks from tjs-lang for *The TJS Programming Language*, a book that teaches
+programming in the page. Single-sourcing for books (#217): one corpus, several outputs,
+nothing kept in sync by hand. And live examples that open with their code showing and a
+console that speaks the example's language (#216). Additive, with one behaviour change to
+know about: in a `tjs` or `ts` example, or one in a registered dialect, a line typed at the
+console is now evaluated in that language and not as JavaScript. `dist/iife.js` grows by
+0.7 kB gzip.
+
+### The book opens on the project, not on "Adopting the doc system"
+
+Both pages are pinned to the top, and the adopting page had an `order` where the README had
+none, so it sorted first: the ePub's opening chapter, and the first entry in the site nav, was
+a page about adopting the doc system. The README now leads both.
+
+### One source, several outputs (#217)
+
+- **Per-volume placement.** A page can sit in a different place in each book:
+  `{"placement": {"language": {"parent": "for-programmers", "order": 20}}}` in its metadata.
+  Each entry may set `parent`, `order`, `pin` and `title`, and naming a volume also binds the
+  page into it. The site nav is unaffected.
+- **Text insets.** `<!--{ "inset": "install.md#with-bun" }-->` includes another document, or
+  one heading's section of it, in the page. A fragment (a markdown file whose name starts
+  with `_`, in a directory the docs are extracted from) can be inset without being a page, and is the only kind of
+  file an inset reads. Resolved at extraction, so the site, the ePubs, `llms.txt` and the markdown copies
+  all get the assembled text, and an inset passage's live examples and tests run where it is
+  inset.
+- **Conditional text.** `<!--{ "only": "book" | "site" }--> … <!--{ "end": "only" }-->`
+  conditional text, for wording that belongs to one output, such as "next chapter" against
+  "next page". Books read `bookText`, everything else reads `text`; both are in the published
+  `docs.json`, so this chooses wording and does not hide anything.
+- **Each is guarded.** An inset that resolves nowhere, an include cycle, a missing heading, a
+  file that is not a fragment, an unbalanced `only`, a directive inside a list, quote or
+  paragraph, and a placement under a parent that is not in the volume all fail the build.
+- **Print is not a volume.** The in-app Print output takes the book wording but stays the
+  whole site in site order; `placement` arranges ePub volumes.
+- New from `tosijs-ui/site`, for a consumer that builds its corpus some other way:
+  `assembleCorpus`, `resolveInsets`, `splitConditions`, `SingleSourceError`, `placeInBook`,
+  `bookTextOf`, and the `BookPlacement` type.
+- Saving an edited live example back to source is refused on a page whose source uses an
+  inset or conditional text, where the example's position on the page no longer identifies a
+  block in that file. No existing page is affected: it applies only to a page that uses one
+  of the new directives.
+
+### Examples that open with their code showing, and a console in their language (#216)
+
+- **A fence picks the view an example opens in.** `{"view": "code"}` in the source block's
+  fence options opens the editor beside the preview, in the page and not full-screen;
+  `{"view": "console"}` puts the console where the preview would be, beside the code, for
+  examples whose point is a value. The editor is built as the example nears the viewport. In
+  a narrow column the panes stack. `example.showInline('code' | 'console')` does the same
+  from code.
+- **The console evaluates in the example's dialect.** A line goes through the same transform
+  as the example's code, with the same fence options, so `x == '5'` in a `tjs` example is
+  TJS's `==`. A line the dialect rejects shows the dialect's error and is not retried as
+  JavaScript. Plain `js` examples and `run` dialects are unchanged. A registered dialect's
+  `transform` now also receives console lines, marked with `options.repl === true`.
+- **Known limit of the above with tjs.** tjs puts setup statements in front of the line it
+  translates, so an expression shows its value, but a line that is only a declaration shows a
+  leftover value where a browser console shows `undefined`, and a line needing both that
+  setup and `await` shows `undefined`. Fixing it needs the setup to be separable, which is
+  asked of tjs-lang.
+- The generated JavaScript tjs-lang asked to see already exists as the read-only **JS** tab of
+  a `tjs` or `ts` example, and the inline views make it reachable in the page. Showing the
+  JavaScript for a single console line is not in this release: a transform's output for one
+  line is mostly its runtime prelude, and there is no general way to separate the two.
+
 ## 1.16.8
 
 A patch for honest results. The doc-test lane can no longer pass over pages that never

package/README.md

@@ -1,6 +1,6 @@
 # tosijs-ui
 
-<!--{ "pin": "top", "layout": "full-width", "headTitle": "tosijs-ui — robust, standards-based web components", "description": "A small, fast library of standards-based web components (data tables, dialogs, forms, rich text, carousels, and more) built on tosijs — smaller and faster than a React app, with no virtual DOM.", "keywords": "web components, custom elements, tosijs, data table, dialog, form, rich text, no virtual dom" }-->
+<!--{ "pin": "top", "order": 0, "layout": "full-width", "headTitle": "tosijs-ui — robust, standards-based web components", "description": "A small, fast library of standards-based web components (data tables, dialogs, forms, rich text, carousels, and more) built on tosijs — smaller and faster than a React app, with no virtual DOM.", "keywords": "web components, custom elements, tosijs, data table, dialog, form, rich text, no virtual dom" }-->
 
 [ui.tosijs.net live demo](https://ui.tosijs.net) | [tosijs](https://tosijs.net) | [discord](https://discord.gg/ramJ9rgky5) | [github](https://github.com/tonioloewald/tosijs-ui#readme) | [npm](https://www.npmjs.com/package/tosijs-ui)
 

package/dist/doc-system/book-html.d.ts

@@ -3,6 +3,8 @@ export interface BookDoc {
     filename: string;
     title: string;
     text: string;
+    /** the book's variant, where conditional text made it differ from the site's */
+    bookText?: string;
     parent?: string;
     hidden?: boolean;
 }

package/dist/doc-system/book-html.js

@@ -9,6 +9,7 @@ force-wrapped code listings, not executed.
 import { renderDocMarkdown } from './render.js';
 import { buildSlugMap } from './routing.js';
 import { buildNavTree } from './nav-tree.js';
+import { bookTextOf } from './book-target.js';
 // Clean book typography; force-wraps code so listings never overflow a page.
 export const DEFAULT_BOOK_CSS = `/* doc-system book stylesheet */
 html { font-size: 100%; }
@@ -149,7 +150,7 @@ export function buildBookHtml(docs, opts) {
     const roots = buildNavTree(visible, slugMap);
     const css = opts.css ?? `${DEFAULT_BOOK_CSS}\n${PRINT_CSS}`;
     const chapters = flatten(roots)
-        .map((node) => `<section class="chapter" id="${slugify(node.doc.filename)}">\n${renderDocMarkdown(stripDocMeta(node.doc.text))}\n</section>`)
+        .map((node) => `<section class="chapter" id="${slugify(node.doc.filename)}">\n${renderDocMarkdown(stripDocMeta(bookTextOf(node.doc)))}\n</section>`)
         .join('\n');
     return `<!DOCTYPE html>
 <html lang="${opts.lang ?? 'en'}">

package/dist/doc-system/book-target.d.ts

@@ -14,6 +14,16 @@ export interface BookTargetDoc {
     /** one book, several books, or `'none'` */
     book?: string | string[];
     hidden?: boolean;
+    /** where this doc sits in a particular volume, keyed by `book` name */
+    placement?: Record<string, BookPlacement>;
+}
+export interface BookPlacement {
+    /** parent within this volume; `''` puts the doc at the top level */
+    parent?: string;
+    order?: number;
+    pin?: 'top' | 'bottom';
+    /** chapter title in this volume */
+    title?: string;
 }
 type SlugMap = Record<string, string>;
 /**
@@ -29,6 +39,15 @@ export declare function chain<T extends BookTargetDoc>(doc: T, docs: T[], slugMa
  * another, or bind into several (`["default", "appendices"]`).
  */
 export declare function resolveBooks<T extends BookTargetDoc>(doc: T, docs: T[], slugMap?: SlugMap): string[];
+/**
+ * The docs of one volume, arranged for it: each doc's `placement` for this book overlaid on
+ * its site `parent` / `order` / `pin` / `title`. Returns shallow copies; the site's
+ * arrangement is untouched.
+ */
+export declare function placeInBook<T extends BookTargetDoc & {
+    order?: number;
+    pin?: 'top' | 'bottom';
+}>(docs: T[], book: string): T[];
 /**
  * Hidden here or anywhere above.
  *
@@ -48,4 +67,9 @@ export declare function withoutHidden<T extends BookTargetDoc>(docs: T[], slugMa
 export declare function partitionByBook<T extends BookTargetDoc>(docs: T[], slugMap?: SlugMap): Map<string, T[]>;
 /** Named books only, in stable order — the extra volumes beside the default one. */
 export declare function namedBooks<T extends BookTargetDoc>(docs: T[], slugMap?: SlugMap): string[];
+export declare const bookTextOf: (doc: {
+    text: string;
+    bookText?: string;
+}) => string;
+export declare const mayHaveSingleSourceDirective: (text: string) => boolean;
 export {};

package/dist/doc-system/book-target.js

@@ -29,6 +29,11 @@ export const NO_BOOK = 'none';
  * `book: ["default", "field-guide"]` binds the doc into both.
  */
 export const DEFAULT_BOOK_NAME = 'default';
+const placementKey = (book) => book === DEFAULT_BOOK ? DEFAULT_BOOK_NAME : book;
+/** Volumes a doc's `placement` names, as book keys. */
+function placedBooks(doc) {
+    return Object.keys(doc.placement ?? {}).map((k) => k.trim().toLowerCase() === DEFAULT_BOOK_NAME ? DEFAULT_BOOK : k.trim());
+}
 /**
  * The doc itself, then each ancestor nearest-first. Stops at an unresolvable parent and
  * at a cycle, so a self-parented doc yields just itself rather than hanging.
@@ -86,14 +91,51 @@ function normalizeDeclaration(value) {
  * another, or bind into several (`["default", "appendices"]`).
  */
 export function resolveBooks(doc, docs, slugMap = {}) {
+    const placed = placedBooks(doc);
+    const withPlaced = (books) => [
+        ...books,
+        ...placed.filter((b) => !books.includes(b)),
+    ];
     for (const d of chain(doc, docs, slugMap)) {
         if (d.book === undefined)
             continue;
         const declared = normalizeDeclaration(d.book);
-        if (declared !== null)
-            return declared;
+        /*
+        An explicit `none` on the doc ITSELF still wins over its own placement: withholding is
+        the conservative reading of a contradiction, as it is for `["default", "none"]`. An
+        inherited `none` does not, or a chapter could never be placed into a book from a section
+        that is otherwise site-only.
+        */
+        if (declared !== null) {
+            if (!declared.length && d === doc)
+                return [];
+            return withPlaced(declared);
+        }
     }
-    return [DEFAULT_BOOK];
+    return withPlaced([DEFAULT_BOOK]);
+}
+/**
+ * The docs of one volume, arranged for it: each doc's `placement` for this book overlaid on
+ * its site `parent` / `order` / `pin` / `title`. Returns shallow copies; the site's
+ * arrangement is untouched.
+ */
+export function placeInBook(docs, book) {
+    const key = placementKey(book);
+    return docs.map((doc) => {
+        const entry = Object.entries(doc.placement ?? {}).find(([k]) => k.trim() === key ||
+            (book === DEFAULT_BOOK && k.trim().toLowerCase() === DEFAULT_BOOK_NAME));
+        if (!entry)
+            return doc;
+        // Only what a placement is documented to set: an entry is author-written JSON, and
+        // copying whatever it holds would let `"hidden": false` or `"text"` through.
+        const out = { ...doc };
+        for (const field of ['parent', 'order', 'pin', 'title']) {
+            const value = entry[1][field];
+            if (value !== undefined)
+                out[field] = value;
+        }
+        return out;
+    });
 }
 /**
  * Hidden here or anywhere above.
@@ -134,3 +176,20 @@ export function namedBooks(docs, slugMap = {}) {
         .filter((k) => k !== DEFAULT_BOOK)
         .sort();
 }
+/*
+The text a book binds for this doc.
+
+`bookText` exists only where conditional text made the book differ from the site
+(`site/single-source.ts`). Every book output asks here, so there are exactly two readers to
+get right (the ePub and print) and every other consumer keeps reading `text`.
+*/
+export const bookTextOf = (doc) => doc.bookText ?? doc.text;
+/*
+Might this source use an inset or conditional text?
+
+Deliberately loose: it also matches a directive that is only being SHOWN in a code fence.
+Its two callers want exactly that bias. The build uses it to skip docs that certainly have
+none before asking the parser, and the example editor uses it to refuse a save, where a
+false positive costs a refusal and a false negative writes an edit into the wrong block.
+*/
+export const mayHaveSingleSourceDirective = (text) => /<!--\s*\{[^\n]*"(inset|only|end)"\s*:/.test(text);

package/dist/doc-system/site/docs.d.ts

@@ -1,3 +1,4 @@
+import { type BookPlacement } from '../book-target.js';
 export interface Doc {
     text: string;
     title: string;
@@ -32,6 +33,18 @@ export interface Doc {
      * individual chapter can still divert, join several volumes, or opt out.
      */
     book?: string | string[];
+    /**
+     * Where this doc sits in a particular volume, when that differs from the site:
+     * `{ "language": { "parent": "for-programmers", "order": 20 } }`. Keyed by `book` name
+     * (`"default"` for the main volume); each entry may set `parent`, `order`, `pin` and
+     * `title`. Naming a volume here also binds the doc into it. The site nav is unaffected.
+     */
+    placement?: Record<string, BookPlacement>;
+    /**
+     * The text a book binds, set by the build only where `{"only": …}` conditional text made
+     * it differ from `text` (which is always what the site shows). Never author this.
+     */
+    bookText?: string;
     /**
      * Opt this page out of the reading column.
      *

package/dist/doc-system/site/docs.js

@@ -99,6 +99,7 @@ import { truncationWarnings, formatTruncationWarnings, } from './truncated-doc.j
 import { pinnedSort } from '../nav-tree.js';
 import { buildSlugMap } from '../routing.js';
 import { withoutHidden } from '../book-target.js';
+import { assembleCorpus, isSingleSourceDirective } from './single-source.js';
 /**
  * Directories a doc site should not publish, excluded by default (tosijs-ui#153).
  *
@@ -234,7 +235,14 @@ function metadata(content, filePath) {
     And it must START A LINE. A metadata block is a standalone directive, so an inline mention
     in a sentence is prose — exactly the rule `/*#` doc blocks already follow.
     */
-    const source = scannable.match(/^[ \t]*<!--(\{.*\})-->|^[ \t]*\/\*(\{.*\})\*\//m);
+    /*
+    The first such block that is not a single-sourcing directive. An inset or an `only` is
+    spelled the same way, and a doc that opens with one and has no metadata of its own was
+    otherwise published carrying `inset` as a metadata field.
+    */
+    const source = [
+        ...scannable.matchAll(/^[ \t]*<!--(\{.*\})-->|^[ \t]*\/\*(\{.*\})\*\//gm),
+    ].find((m) => !isSingleSourceDirective(m[1] || m[2]));
     let data = {};
     if (source) {
         try {
@@ -297,7 +305,9 @@ function hasDocBlock(filePath) {
         return false; // unreadable is not our business here
     }
 }
-function findMarkdownFiles(paths, ignore, defaultIgnores = new Set()) {
+function findMarkdownFiles(paths, ignore, defaultIgnores = new Set(), 
+/** filled with the real path of every directory actually walked (see `assembleCorpus`) */
+walked) {
     const markdownFiles = [];
     const truncationFound = [];
     const withheld = [];
@@ -366,6 +376,12 @@ function findMarkdownFiles(paths, ignore, defaultIgnores = new Set()) {
             }
             return;
         }
+        try {
+            walked?.add(fs.realpathSync(dir));
+        }
+        catch {
+            // unreadable: nothing in it is scraped, so nothing in it is a fragment either
+        }
         files.forEach((file) => {
             // Skip scaffolding / working files (Jekyll-style `_`-prefix): a
             // `_template.md` or `_drafting-log.md` shouldn't leak into the corpus or
@@ -498,7 +514,8 @@ function findMarkdownFiles(paths, ignore, defaultIgnores = new Set()) {
 }
 export function extractDocs(options) {
     const { paths, ignore = ['node_modules', 'dist', 'build'], output } = options;
-    const found = findMarkdownFiles(paths, ignore, new Set(DEFAULT_DOC_IGNORES));
+    const walked = new Set();
+    const found = findMarkdownFiles(paths, ignore, new Set(DEFAULT_DOC_IGNORES), walked);
     /*
     Drop hidden docs HERE, before anything else sees them.
   
@@ -513,6 +530,21 @@ export function extractDocs(options) {
     rest of the pipeline is concerned. Descendants of a hidden doc go with it.
     */
     const docs = withoutHidden(found, buildSlugMap(found));
+    /*
+    Insets and conditional text are resolved HERE, for the same reason hidden docs are dropped
+    here: the corpus is the thing that ships, so anything assembled later is assembled by each
+    consumer separately. After the hidden filter, so a withheld doc cannot be published by
+    insetting it; a fragment that should not be a page is a `_`-prefixed file instead, and that
+    is the only kind of file an inset will read (see `fragmentPath`).
+    */
+    /*
+    A fragment may be read from a directory this extraction WALKED, and nowhere else. Not a
+    list of roots with the ignore rule applied again: the re-review found that second copy
+    disagreeing with the first within the hour (a file doc path such as `README.md` made the
+    whole project a root, and a path-form ignore was not honoured). The walk already decided
+    which directories are published, so ask it.
+    */
+    assembleCorpus(docs, { fragmentDirs: walked });
     if (output) {
         saveDocsJSON(docs, output);
     }

package/dist/doc-system/site/epub.js

@@ -18,9 +18,9 @@ import * as fs from 'fs';
 import * as path from 'path';
 import { renderDocMarkdown } from '../render.js';
 import { highlightHtml } from '../highlight.js';
-import { buildSlugMap, pathForSlug, slugForPath, withBase } from '../routing.js';
+import { buildSlugMap, pathForSlug, resolveParent, slugForPath, withBase, } from '../routing.js';
 import { buildNavTree } from '../nav-tree.js';
-import { partitionByBook, DEFAULT_BOOK } from '../book-target.js';
+import { partitionByBook, placeInBook, bookTextOf, DEFAULT_BOOK, } from '../book-target.js';
 import { epubVolumeIdentity } from './epub-volumes.js';
 import { isLiveFence, languageOfClass } from '../example-policy.js';
 import { DEFAULT_BOOK_CSS, stripDocMeta, flatten, slugify, } from '../book-html.js';
@@ -638,8 +638,28 @@ export async function buildEpub(config, opts = {}) {
             `   A doc joins a volume with \`{"book": "<name>"}\` (inherited by its children).\n` +
             `   If this is a typo, note that book names are case- and space-sensitive.`);
     }
+    /*
+    Arrange for THIS volume before curating: a doc's `placement` for this book replaces its
+    site parent and order (#217), and the manifest then orders what results.
+  
+    A placement parent that is not in the volume is an error. The nav tree would otherwise
+    quietly promote the chapter to the top level, which is a book with the wrong outline and a
+    green build.
+    */
+    const book = opts.bookTarget ?? DEFAULT_BOOK;
+    const placed = placeInBook(visible, book);
+    const placedSlugs = buildSlugMap(placed);
+    for (const [i, doc] of placed.entries()) {
+        if (doc === visible[i] || !doc.parent)
+            continue;
+        if (doc.parent === visible[i].parent)
+            continue;
+        if (!resolveParent(doc.parent, placed, placedSlugs)) {
+            throw new Error(`ePub volume ${JSON.stringify(book || '(default)')}: ${doc.filename} is placed under "${doc.parent}", which is not a document in that volume.`);
+        }
+    }
     // Curate/reorder for the book (a no-op when config.book is absent).
-    const docs = selectBookDocs(visible, config.book);
+    const docs = selectBookDocs(placed, config.book);
     const slugMap = buildSlugMap(docs);
     const roots = buildNavTree(docs, slugMap);
     const fileFor = (d) => `${slugMap[d.filename] || 'index'}.xhtml`;
@@ -727,7 +747,7 @@ export async function buildEpub(config, opts = {}) {
         `injectExampleLinks` matches on the `language-*` class, never on the code text, so
         highlighting cannot break the "Run this example live" links.
         */
-        await highlightHtml(renderDocMarkdown(stripDocMeta(doc.text)), 'none'), bookFiles, slugMap, config.basePath);
+        await highlightHtml(renderDocMarkdown(stripDocMeta(bookTextOf(doc))), 'none'), bookFiles, slugMap, config.basePath);
         // happy-dom occasionally throws on exotic content (e.g. an internal selector
         // bug); fall back to the regex pass for that doc rather than aborting.
         let bodyHtml;

package/dist/doc-system/site/index.d.ts

@@ -18,6 +18,9 @@ export { buildEpub, DEFAULT_BOOK_CSS } from './epub.js';
 export type { BuildEpubOptions, BookMeta } from './epub.js';
 export { selectBookDocs } from '../book-manifest.js';
 export type { BookManifest } from '../book-manifest.js';
+export { placeInBook, bookTextOf } from '../book-target.js';
+export type { BookPlacement } from '../book-target.js';
+export { assembleCorpus, resolveInsets, splitConditions, SingleSourceError, } from './single-source.js';
 export { listEpubVolumes, epubVolumeIdentity, renderEpubDownloads, type EpubVolume, } from './epub-volumes.js';
 export { currentHolder, describeHolder, lockPathFor, lockDecision, isProcessAlive, } from './build-lock.js';
 export type { LockHolder, LockDecision } from './build-lock.js';

package/dist/doc-system/site/index.js

@@ -18,6 +18,10 @@ export { extractDocs, saveDocsJSON } from './docs.js';
 export { generateSite } from './generate-site.js';
 export { buildEpub, DEFAULT_BOOK_CSS } from './epub.js';
 export { selectBookDocs } from '../book-manifest.js';
+export { placeInBook, bookTextOf } from '../book-target.js';
+// Insets and conditional text (#217): `extractDocs` applies these; exported for a consumer
+// that builds its corpus some other way.
+export { assembleCorpus, resolveInsets, splitConditions, SingleSourceError, } from './single-source.js';
 export { listEpubVolumes, epubVolumeIdentity, renderEpubDownloads, } from './epub-volumes.js';
 /*
 Reading the build lock is part of the PUBLIC surface, not an internal detail (#117).

package/dist/doc-system/site/orchestrator.js

@@ -643,6 +643,14 @@ export async function buildSite(config, opts = {}) {
                         doc.text = filled;
                         touched++;
                     }
+                    // The book's variant is a second copy of the text (conditional text, #217).
+                    if (typeof doc.bookText === 'string') {
+                        const book = renderEpubDownloads(doc.bookText, volumes);
+                        if (book !== doc.bookText) {
+                            doc.bookText = book;
+                            touched++;
+                        }
+                    }
                 }
                 if (touched)
                     await Bun.write(DOCS_JSON, JSON.stringify(corpus, null, 2));

package/dist/doc-system/site/single-source.d.ts

@@ -0,0 +1,31 @@
+export declare class SingleSourceError extends Error {
+    constructor(message: string);
+}
+interface SourceDoc {
+    text: string;
+    filename: string;
+    path?: string;
+    title?: string;
+    /** the text a book binds, when it differs from `text` (the site's) */
+    bookText?: string;
+}
+/** Is this JSON directive single-sourcing rather than doc metadata? */
+export declare function isSingleSourceDirective(json: string): boolean;
+export interface ResolveInsetsOptions {
+    /**
+     * Real paths of the directories a fragment file may be read from: the ones extraction
+     * walked. `extractDocs` supplies this. Without it no file inset is read at all.
+     */
+    fragmentDirs?: ReadonlySet<string>;
+    /** read a fragment file; injectable so the rule is testable without a filesystem */
+    readFile?: (absolutePath: string) => string | undefined;
+    /** resolve symlinks; injectable for the same reason */
+    realPath?: (absolutePath: string) => string;
+}
+export declare function resolveInsets(docs: SourceDoc[], options?: ResolveInsetsOptions): number;
+export declare function splitConditions(text: string, where?: string): {
+    site: string;
+    book: string;
+};
+export declare function assembleCorpus(docs: SourceDoc[], options?: ResolveInsetsOptions): void;
+export {};