@docsworth/cli 0.2.0 → 0.3.0

package/README.md

@@ -44,7 +44,8 @@ docsworth init --team platform     # config, CODEOWNERS, root map, a guide
 docsworth lint .                   # 3 pages, 0 errors, 0 warnings
 ```
 
-Add `--ci` to `init` for a workflow that lints and compiles on every PR.
+`init` reads your GitHub org from the repo's remote; in a repo without one yet, add
+`--org <your-org>`. Add `--ci` for a workflow that lints and compiles on every PR.
 
 Install the Claude Code plugin from this repo's marketplace:
 
@@ -67,12 +68,54 @@ Start Claude Code in the knowledge repo, or set `DOCSWORTH_KNOWLEDGE_DIR` to its
 The server uses your own GitHub identity (`GITHUB_TOKEN`, `GH_TOKEN` or `gh auth
 token`), so your repo and team permissions decide what you see.
 
+## Adopting existing docs
+
+Docs you already have, a Docusaurus site's say, become pages without moving a URL. With
+them under `knowledge/` (or `knowledge_root` in `.knowledge/config.yaml` set to where they
+are), after `init`:
+
+```bash
+docsworth adopt --team docs        # the frontmatter each page lacks, as drafts
+docsworth lint .
+```
+
+`adopt` gives each page the fields it lacks, and keeps everything it has:
+
+- a page's `id` is its own, else its file name, a Docusaurus number prefix removed, so the
+  site's URLs stay where they are;
+- its owner comes from CODEOWNERS, else `--team`;
+- a route link like `/guides/setup` becomes a relative link to the file,
+  `../guides/setup.md`;
+- keys a site generator keeps, such as `sidebar_position`, go in `frontmatter.extra_fields`;
+- images a site serves from its `static/` directory (`/img/x.png`) get a hint naming the
+  `static_dirs` line to add, after which lint reads them there.
+
+A page or link it can't settle stays as it is, listed, and lint fails it until it's fixed
+by hand: a category index like `guides/index.md`, two pages with one file name, a link to
+nothing. `--dry-run` reports and writes nothing.
+
+Adopted pages are drafts, so agents cite none of them yet. Once a team has read its pages:
+
+```bash
+docsworth verify knowledge/guides  # canonical, verified today
+```
+
+Both commands only change files. Commit them and open a pull request, which CODEOWNERS
+sends to the owning team.
+
+If the site stays up, set `site_url` (`https://docs.example.com/docs/`) in the config.
+Then a link to a page someone can't see reads `[restricted page]` by its full URL on the
+site, as it already does by its route. A link without an extension that names nothing
+reads `[restricted page]` too: a route can't be told from a hidden page's.
+
 ## Commands
 
 | Command | What it does |
 | --- | --- |
 | `docsworth init [dir]` | Scaffold config, CODEOWNERS, `knowledge/index.md`, a glossary and a writing guide. Never overwrites. |
 | `docsworth lint [dir]` | Frontmatter, CODEOWNERS ownership, links and duplicate ids. Exit 1 on errors; overdue pages only warn. |
+| `docsworth adopt [dir] [--team t]` | Existing pages get the frontmatter they lack, as drafts, and route links become `.md` links; lists what it can't settle. Working tree only; `--dry-run`. |
+| `docsworth verify <paths…>` | The draft or canonical pages under the paths become canonical, verified today. Working tree only; `--dry-run`. |
 | `docsworth stale [dir]` | Pages past their review SLA or without a team owner. |
 | `docsworth coverage [dir] [--html f]` | Percentage team-owned and percentage verified within SLA, by team; optional static dashboard. |
 | `docsworth digest [dir] [--send]` | Weekly per-team digest: due, overdue, most read by agents. Prints unless `--send`. |
@@ -131,11 +174,20 @@ npm run docsworth -- --as-of 2026-11-02 audit examples/acme --out audit
 ## Browser extension
 
 A side panel for Chrome (and other Chromium browsers) over `docsworth serve`. From the panel
-you can search and read pages, or suggest an addition to one. From any page, right-click a
-selection and choose **Save selection to Docsworth**. In Gmail, a **Save to Docsworth**
-button beside a thread's subject opens the panel. The panel first shows the pages that may
-already cover the thread, then a draft page filled from it. Nothing is written until you
-open the draft PR. It isn't in the Chrome Web Store yet: build it and load it unpacked.
+you can search and read pages, or suggest an addition to one.
+
+**Saving what you read.**
+- **From the menu:** on any page, right-click and choose **Save page to Docsworth**, or
+  select some text and choose **Save selection to Docsworth**.
+- **From the panel:** **Save this page** and **Save the selection** read the tab you opened
+  the panel from with the toolbar button.
+- **In Gmail:** a **Save to Docsworth** button beside a thread's subject.
+
+The page's main content or the selection arrives as markdown, with its headings, links,
+code blocks and tables, and without what the page hides or its navigation. The panel first
+shows the pages that may already cover it, each with **Add the capture here**, then a draft
+page filled from it, with its source. Nothing is written until you open the PR. The
+extension isn't in the Chrome Web Store yet: build it and load it unpacked.
 
 ```bash
 npm run build -w @docsworth/extension
@@ -177,8 +229,8 @@ See [AGENTS.md](AGENTS.md), and [docs/releasing.md](docs/releasing.md) for relea
 
 ## Status
 
-v0.2. What's in it, and what hasn't been verified against the real thing yet, is in
-[the release notes](docs/release-notes/v0.2.0.md).
+v0.3. What's in it, and what hasn't been verified against the real thing yet, is in
+[the release notes](docs/release-notes/v0.3.0.md).
 
 ## License
 

package/dist/cli.js

@@ -5,6 +5,7 @@ import { dirname, join, resolve } from "node:path";
 import { fileURLToPath } from "node:url";
 import { Command, InvalidArgumentError } from "commander";
 import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
+import { adoptRepo } from "./core/adopt.js";
 import { actorOf, createAuditLog, stateDir } from "./core/audit-log.js";
 import { CONFIG_PATH, ConfigError, loadConfig } from "./core/config.js";
 import { computeCoverage, formatCoverage } from "./core/coverage.js";
@@ -21,6 +22,8 @@ import { DEFAULT_PORT, startServer } from "./core/serve/server.js";
 import { loadToken } from "./core/serve/token.js";
 import { KnowledgeService } from "./core/service.js";
 import { listStale } from "./core/stale.js";
+import { printable, printableJson } from "./core/printable.js";
+import { verifyPages } from "./core/verify-pages.js";
 import { PACKAGE_VERSION } from "./core/version.js";
 import { existsSync, readFileSync } from "node:fs";
 import { parse as parseYaml } from "yaml";
@@ -172,9 +175,55 @@ program
     .option("--json", "machine-readable output")
     .action((dir, opts) => {
     const result = lintRepo(repoAt(dir), now());
-    console.log(opts.json ? JSON.stringify(result, null, 2) : formatDiagnostics(result));
+    console.log(opts.json ? printableJson(result) : formatDiagnostics(result));
     process.exitCode = result.errors > 0 ? 1 : 0;
 });
+program
+    .command("adopt")
+    .description("give existing markdown pages the frontmatter Docsworth needs, as drafts, and turn route links into .md links (working tree only)")
+    .argument("[dir]", "knowledge repo root", ".")
+    .option("--team <team>", "owning team for pages CODEOWNERS gives no single team")
+    .option("--review-every <period>", "review period for pages without one", "90d")
+    .option("--dry-run", "report what would change; write nothing")
+    .option("--json", "machine-readable output")
+    .action((dir, opts) => {
+    const r = adoptRepo(resolve(dir), { now: now(), team: opts.team, reviewEvery: opts.reviewEvery, dryRun: opts.dryRun });
+    if (opts.json)
+        return console.log(printableJson(r));
+    // Paths, links, keys and reasons quote what pages hold: printed escaped.
+    const say = (line) => console.log(printable(line));
+    for (const l of r.left)
+        say(`left       ${l.path}: ${l.reason}`);
+    for (const l of r.linksLeft)
+        say(`link left  ${l.path}: \`${l.link}\` ${l.why}`);
+    if (r.extraFields.length)
+        say(`config     frontmatter.extra_fields gains ${r.extraFields.join(", ")}`);
+    for (const h of r.hints)
+        say(`hint       ${h}`);
+    const n = (count, what) => `${count} ${what}${count === 1 ? "" : "s"}`;
+    console.log(`\n${r.dryRun ? "Would adopt" : "Adopted"} ${n(r.adopted.length, "page")} (${r.unchanged.length} unchanged, ${r.left.length} left); ` +
+        `${r.linksRewritten} ${r.linksRewritten === 1 ? "link" : "links"} rewritten, ${r.linksLeft.length} left.`);
+    console.log(r.dryRun
+        ? "Dry run: nothing written."
+        : `Nothing is committed. Review the diff, run \`docsworth lint ${dir}\`, and open a pull request: adopted pages are drafts, cited once their team runs \`docsworth verify\`.`);
+});
+program
+    .command("verify")
+    .description("verify pages: each draft or canonical page under the paths becomes canonical, verified today (working tree only)")
+    .argument("<paths...>", "pages, or directories of them")
+    .option("--dir <dir>", "knowledge repo root", ".")
+    .option("--dry-run", "report what would change; write nothing")
+    .option("--json", "machine-readable output")
+    .action((paths, opts) => {
+    // Paths are the shell's, relative to where it runs, whatever --dir says.
+    const r = verifyPages(resolve(opts.dir), paths.map((p) => resolve(p)), { now: now(), dryRun: opts.dryRun });
+    if (opts.json)
+        return console.log(printableJson(r));
+    for (const l of r.left)
+        console.log(printable(`left  ${l.path}: ${l.reason}`));
+    console.log(`\n${r.dryRun ? "Would verify" : "Verified"} ${r.verified.length} page${r.verified.length === 1 ? "" : "s"} (${r.unchanged.length} already verified today, ${r.left.length} left).`);
+    console.log(r.dryRun ? "Dry run: nothing written." : "Nothing is committed: open a pull request, which the owning team reviews.");
+});
 program
     .command("stale")
     .description("pages past their review SLA or without a team owner")
@@ -295,7 +344,6 @@ program
     const web = join(dirname(fileURLToPath(import.meta.url)), "web");
     const serving = await startServer({
         root,
-        knowledgeRoot: config.knowledge_root,
         identity,
         who,
         audit,
@@ -400,7 +448,8 @@ const isMain = process.argv[1] && /(?:cli\.(?:ts|js)|docsworth)$/.test(process.a
 if (isMain) {
     program.parseAsync().catch(async (err) => {
         if (isUserError(err)) {
-            console.error(`docsworth: ${err.message}`);
+            // A message can quote a path or a value a page holds: printed escaped.
+            console.error(`docsworth: ${printable(err.message)}`);
             // Exit 3 if the failure also left a change its required audit log doesn't show.
             process.exit(process.exitCode === EXIT_AUDIT_DEGRADED ? EXIT_AUDIT_DEGRADED : 2);
         }

package/dist/core/acl.d.ts

@@ -26,3 +26,12 @@ export declare const RESTRICTED = "[restricted page]";
  * definitions, and HTML anchors.
  */
 export declare function redactLinks(body: string, fromPath: string, repo: Repo, who: Identity): string;
+/** One pass of the sweep for `who` on page `fromPath`: `liberal`, every route naming nothing fails closed, a link or not. */
+export declare function redactionSweep(fromPath: string, repo: Repo, who: Identity, gone?: Set<string>): (text: string, liberal: boolean) => string;
+/**
+ * `sweep` run until it changes nothing. Replacing a line can change what the rest means (a
+ * fence gone, code is markdown, and its links are links), so a pass can leave work for the
+ * next. Past `passes`, one last `liberal` pass, where every route naming nothing fails
+ * closed, ends it.
+ */
+export declare function untilStable(text: string, sweep: (text: string, liberal: boolean) => string, passes?: number): string;

package/dist/core/acl.js

@@ -1,5 +1,5 @@
 import { ACL_INTERNAL } from "./schema.js";
-import { destinationHides, namingLines, pageNames, replaceLines } from "./references.js";
+import { definitionsIn, destinationHides, linkPaths, normalLabel, namingLines, pageNames, replaceLines } from "./references.js";
 export function canSee(page, who) {
     if (!who.repoAccess)
         return false;
@@ -38,8 +38,8 @@ const A_CLOSE = /<\/a\s*>/i;
  * path. Anything under the knowledge root that is not a loadable page fails closed: an
  * invalid page has no trustworthy ACL.
  */
-function hidesPage(dest, fromPath, repo, who, html = false) {
-    return destinationHides(dest, fromPath, namesFor(repo, who), html);
+function hidesPage(dest, fromPath, repo, who, html = false, links) {
+    return destinationHides(dest, fromPath, namesFor(repo, who), html, links);
 }
 /**
  * Rewrite every link to a page the caller cannot see, so a visible page (say the root
@@ -48,26 +48,60 @@ function hidesPage(dest, fromPath, repo, who, html = false) {
  * definitions, and HTML anchors.
  */
 export function redactLinks(body, fromPath, repo, who) {
-    const hidden = (dest) => hidesPage(dest, fromPath, repo, who);
+    // A route naming nothing fails closed where CommonMark reads a link (references.ts).
+    const links = linkPaths(body);
+    const hidden = (dest) => hidesPage(dest, fromPath, repo, who, false, links);
     // Reference definitions first: remember which labels point at hidden pages.
     const hiddenLabels = new Set();
+    // Only the definition CommonMark puts in effect for a label (code review, round 9): a
+    // `]:` inside a paragraph defines nothing, and is left for the sweep.
+    const defs = definitionsIn(body);
     let out = body.replace(REF_DEF, (whole, label, dest) => {
         if (!hidden(dest))
             return whole;
+        const inEffect = defs?.get(normalLabel(label));
+        if (defs && (inEffect === undefined || !hidden(inEffect)))
+            return whole;
         hiddenLabels.add(label.trim().toLowerCase());
         return "";
     });
     if (hiddenLabels.size) {
         // [text][label], [label][] and the shortcut form [label].
         out = out.replace(/\[([^\][]{0,999})\]\[([^\][]{0,999})\]/g, (whole, text, label) => hiddenLabels.has((label || text).trim().toLowerCase()) ? RESTRICTED : whole);
-        out = out.replace(/\[([^\][]{1,999})\](?![(\[:])/g, (whole, label) => (hiddenLabels.has(label.trim().toLowerCase()) ? RESTRICTED : whole));
+        // Not a label right after a `]`: that's a full reference whose text the form above
+        // couldn't read (it holds brackets: merge gate, PR 53, round 3), and the label alone
+        // would leave its text. The sweep takes those lines whole.
+        out = out.replace(/(?<!\])\[([^\][]{1,999})\](?![(\[:])/g, (whole, label) => (hiddenLabels.has(label.trim().toLowerCase()) ? RESTRICTED : whole));
     }
     out = out.replace(INLINE, (whole, _text, dest) => (hidden(dest) ? RESTRICTED : whole));
     out = redactAnchors(out, (href) => hidesPage(href, fromPath, repo, who, true));
     // The passes above keep the rest of a line, but read only the common forms. The sweep
     // after them fails closed: a line that still names a hidden page, in any form a reader
     // resolves, goes whole (references.ts).
-    return replaceLines(out, namingLines(out, fromPath, namesFor(repo, who)), RESTRICTED);
+    // The labels the first pass took the definitions of: the sweep takes their references,
+    // which the forms above miss (`[ref]:` that can't start a definition is one).
+    return untilStable(out, redactionSweep(fromPath, repo, who, new Set(hiddenLabels)));
+}
+/** One pass of the sweep for `who` on page `fromPath`: `liberal`, every route naming nothing fails closed, a link or not. */
+export function redactionSweep(fromPath, repo, who, gone = new Set()) {
+    const names = namesFor(repo, who);
+    // `gone`: the labels the first pass took the definitions of, whose references each pass
+    // takes too.
+    return (text, liberal) => replaceLines(text, namingLines(text, fromPath, names, 0, [], liberal, gone), RESTRICTED);
+}
+/**
+ * `sweep` run until it changes nothing. Replacing a line can change what the rest means (a
+ * fence gone, code is markdown, and its links are links), so a pass can leave work for the
+ * next. Past `passes`, one last `liberal` pass, where every route naming nothing fails
+ * closed, ends it.
+ */
+export function untilStable(text, sweep, passes = 3) {
+    for (let pass = 1;; pass++) {
+        const next = sweep(text, pass > passes);
+        if (next === text || pass > passes)
+            return next;
+        text = next;
+    }
 }
 /** Each `<a href="…">…</a>` whose href is hidden, as the placeholder (its `</a>` within 4,000 characters). */
 function redactAnchors(text, hidden) {

package/dist/core/adopt.d.ts

@@ -0,0 +1,68 @@
+import { type Config } from "./config.js";
+import { UserError } from "./errors.js";
+import { unprefixed } from "./links.js";
+export { unprefixed };
+export declare class AdoptError extends UserError {
+}
+export interface AdoptOptions {
+    now: Date;
+    /** The owning team where CODEOWNERS doesn't name exactly one. */
+    team?: string;
+    /** `review_every` for pages without one (default 90d). */
+    reviewEvery?: string;
+    /** Report, and write nothing. */
+    dryRun?: boolean;
+}
+export interface AdoptResult {
+    dryRun: boolean;
+    /** Pages adopt changed (or would, on a dry run). */
+    adopted: string[];
+    /** Pages it had nothing to change on. */
+    unchanged: string[];
+    /** Pages, and symbolic links, it left as they are, and why. */
+    left: {
+        path: string;
+        reason: string;
+    }[];
+    /** What the team still has to do: a CODEOWNERS line, say. */
+    hints: string[];
+    /** Route links rewritten to relative `.md` links (R2). */
+    linksRewritten: number;
+    /** Links lint fails that adopt couldn't rewrite, as written, and why. */
+    linksLeft: {
+        path: string;
+        link: string;
+        why: string;
+    }[];
+    /** Keys added to the config's `frontmatter.extra_fields`: the adopted pages' unknown keys (R3). */
+    extraFields: string[];
+}
+/** The repo, its config and its knowledge root, by real path, each checked to be inside the repo (R9). */
+export interface RepoPaths {
+    repo: string;
+    config: Config;
+    configPath: string;
+    knowledge: string;
+}
+/** Whether `child` is `parent` or under it, by path components (so `/repo-other` isn't under `/repo`). */
+export declare function within(parent: string, child: string): boolean;
+/** R9's checks, before any page is read: the config and the knowledge root are inside the repo. */
+export declare function repoPaths(root: string, Err?: new (m: string) => Error, verb?: string): RepoPaths;
+/**
+ * Why a file with other hard links is left, or null: the other path could be anywhere,
+ * outside the repo too, and a write here would change the file there (R9).
+ */
+export declare function hardLinked(abs: string, verb: string): string | null;
+/** Why a symbolic link in the knowledge root is left. */
+export declare const symlinkReason: (verb: string) => string;
+/**
+ * The knowledge root's pages, as the loader finds them (directories starting with a dot
+ * skipped), by real path. It follows no symbolic link: the loader reads none, and a link is
+ * the one way a walk could leave the root (R9). A link that names a page or a directory is
+ * reported.
+ */
+export declare function walk(dir: string, out: {
+    files: string[];
+    symlinks: string[];
+}): void;
+export declare function adoptRepo(root: string, opts: AdoptOptions): AdoptResult;