@doxbrix/doxloop 0.1.4 → 0.2.0

package/docs/generation-performance.md

@@ -0,0 +1,108 @@
+# Generation performance and usage budgets
+
+Planning saves useful browser screenshots while it explores the application.
+The application research checkpoint records capture IDs, paths, visible states,
+labels, alt text and review checks. The approved plan references those IDs in
+`visuals.captureIds`, in the same order as `captureSequence`. A missing capture
+uses an empty ID. New plans prefer meaningful saved images over fixed image
+counts per page type. Explicit user requirements remain authoritative.
+
+Generation copies selected, reviewed PNGs into the generator's asset directory.
+Missing, corrupt, unreviewed or ambiguous captures remain pending. Missing states
+run in short capture-only sessions before writing. Writers receive a compact
+plan slice, relevant source excerpts, extracted English UI labels and saved
+image descriptions. They choose and embed the images without browsing again.
+Older plans without a capture catalogue use the separate capture stage.
+
+Completed writing is checkpointed independently of capture and validation.
+Resume retains written pages and verified captures; validation defects are sent
+to targeted repairs. A crashed child can retry without cancelling the entire
+run. An account-limit response stops further scheduling and retries immediately.
+
+## Limits
+
+A plan shares one persistent usage ledger across planning, capture, writing,
+repairs and resumed attempts: `.doxloop/plans/<id>/usage-budget.json`.
+There is **no limit by default**: a run only stops when a cap is set
+explicitly. Set `sync.budget.maxUsd` in the project, or the variables below,
+to enforce one. The first reached limit stops active sessions and preserves
+work.
+
+Environment overrides, read when Doxloop starts a stage:
+
+| Variable | Default | Meaning |
+| --- | ---: | --- |
+| `DOXLOOP_MAX_TOKENS` | unlimited | Total reported tokens for this plan, across attempts |
+| `DOXLOOP_MAX_USD` | unlimited | Total reported dollars; overrides the project setting |
+| `DOXLOOP_AUTHORING_PARALLEL` | 3 | Concurrent writer sessions (the first batch of a new site runs alone) |
+| `DOXLOOP_AUTHORING_EFFORT` | plan's effort | Reasoning effort of the writer sessions (`low`…`max`) |
+| `DOXLOOP_SUPPORT_EFFORT` | low | Reasoning effort of capture, fix, and retake sessions |
+| `DOXLOOP_CAPTURE_GUIDES_PER_SESSION` | 3 | Guides per capture session (one sign-in each) |
+| `DOXLOOP_AUTHORING_BATCH_PAGES` | 4 | Pages per writer session |
+| `DOXLOOP_PLANNING_PARALLEL` | 2 | Concurrent research sessions (Gemini runs serially) |
+
+Limits are enforced when agents report usage. An in-flight response may
+overshoot; providers that report cost only on completion cannot be stopped
+mid-response by cost alone. These counters are not the provider's five-hour
+subscription allowance. Work done before this ledger existed is not backfilled.
+A quota pause can resume after the provider reset. A Doxloop budget pause requires
+explicitly raising its limit; resume does not reset the ledger.
+
+New starter plans default to 5 pages and 15 captures. Standard and
+comprehensive plans have no page cap: the evidence sets their size, up to the
+300-capture run maximum. A reviewer can still set a cap on the plan review's
+batch-limits panel. Approved plans keep their existing scope. Reducing parallelism limits simultaneous spend, but does
+not by itself reduce total tokens.
+
+## Comparing performance
+
+Compare identical source snapshots, approved pages, capture states, model and
+effort. Measure planning and authoring wall time separately, total reported
+input/cache/output tokens, reported cost, missing-state browser calls, and final
+validation outcome. Exclude idle time between attempts. Local fake-agent tests
+verify reuse and recovery without consuming model allowance; a real-agent
+comparison is needed before claiming a percentage saving.
+
+Offline prompt comparison on the saved `vikunja7` first six-page batch:
+the previous full-request construction was 40,000 bytes; the compact batch
+construction was 10,208 bytes (74% smaller). This comparison holds page scope
+constant and excludes tool results, source excerpts and model completions. It
+is a prompt-size measurement, not a claim of 74% lower billed usage or latency.
+
+## Verifying a change without a run
+
+`doxloop replay <run-directory>` copies a recorded run's workspace
+(`.doxloop/runs/<run-id>`) and re-runs the end-of-generation pipeline over it:
+capture status from the images on disk, the tolerant screenshot check, the
+deterministic post-pass, starter cleanup, and the validation "Accept all"
+performs. It prints every repair and the errors left, exits 1 when the accept
+check would still fail, and never modifies the run folder. Keep the run
+folders of runs that ended blocked: each is a regression fixture for the gate
+that blocked it.
+
+
+## Planning research is scoped to the request
+
+Planning research (product audit, application exploration, existing-site
+audits) is the largest cost of an update run. Before any session starts,
+Doxloop triages the update request:
+
+| Request | Research | Planner |
+|---|---|---|
+| Navigation, icons, ordering, group names, branding, metadata | none | one session, given the current navigation and the icon names the generator can draw; returns preserved pages plus `workspaceInstructions` |
+| Names existing pages (by file name or path) | product audit focused on those pages; application only when screenshots are required | one session from the focused brief |
+| Product-wide, or adds pages on subjects not yet documented | full research, as a create run | one session from every brief |
+
+Deterministic rules decide the clear cases. An ambiguous request gets one
+short triage session that reads nothing and answers from the request and the
+page list; if it fails, the full research runs. The decision is stored on the
+plan (`research`) and shown as the first review advisory.
+
+Briefs are keyed by source snapshot, capture intent, agent, model, and focus,
+not by plan id, so a later plan on unchanged sources borrows the newest saved
+product and existing-documentation briefs instead of auditing again. The
+application brief is never borrowed: it names captures under its own plan.
+
+Generation applies `workspaceInstructions` in one short session after the
+pages are written, editing only the navigation configuration (or theme and
+brand files) and never page content.

package/docs/generator-authoring.md

@@ -12,11 +12,37 @@ Export a default adapter created with `defineGenerator` from
 - package identity and version;
 - authoring skill name and directory;
 - content root, page extensions, ignored output, and content format;
+- native navigation files used to translate a generator-neutral documentation
+  plan into the framework's own sidebar, menu, or index;
 - native build command and output directory;
 - scaffold, preview, and validation implementations;
 - an optional local-asset resolver for root-relative public assets outside the
-  content directory; and
-- an optional page reader for non-Markdown formats.
+  content directory;
+- an optional page reader for non-Markdown formats;
+- an optional `writeNavigation` hook that adds, removes, or renames a page in
+  the generator's own navigation file (MkDocs edits `nav` in `mkdocs.yml`
+  through the YAML document model so comments and Python tags survive;
+  Markdoc rewrites `navigation.json`); and
+- an optional `renderPage` hook that returns HTML for one page without the
+  native build, for previews (the static HTML adapter returns the page's
+  `<main>` landmark);
+- optional `readNavigationTree` and `writeNavigationTree` hooks that expose
+  the whole navigation as a tree of page and group nodes, so the control
+  center's **Navigation** view can reorder, group, and relabel it (MkDocs maps
+  its `nav` list, including `Title: path` and `Section: [...]` entries, and
+  writes it back through the YAML document model);
+- an optional `project.assets` location naming the directory uploads land in
+  and the public prefix pages use to reference them (the control center
+  otherwise uses the generator's conventional directory, such as `static/img`
+  for Docusaurus or `docs/assets` for MkDocs); and
+- an optional `theme.configFile` naming the file the **Branding** panel should
+  point people at, since only the built-in Doxbrix theme is edited in place.
+
+Adapters without `writeNavigation` are treated as file-system driven: Doxloop
+does not edit their configuration. Adapters without the tree hooks are shown
+read-only in the navigation editor with the name of their navigation file.
+Adapters without `renderPage` fall back to the control center's generic
+Markdown preview.
 
 Package metadata should include:
 
@@ -31,14 +57,38 @@ Package metadata should include:
 }
 ```
 
+Adapters declare planning boundaries without implementing a second plan
+schema:
+
+```ts
+planning: {
+  navigationFiles: ['framework.config.js', 'sidebar.json'],
+}
+```
+
+`navigationFiles` are project-relative and identify files an approved plan may
+require the authoring agent to update. Page grouping and ordering remain in the
+core generator-neutral plan; the format skill translates them into native
+syntax.
+
 ## Responsibilities
 
 Scaffolding creates a minimal native project with explicit starter markers that
-must fail release validation until authoring replaces them. Preview runs the
+must fail release validation until authoring replaces them (`<!-- doxloop:starter-page -->`
+in Markdown and HTML, `{/* doxloop:starter-page */}` in MDX, and the
+`.. doxloop:starter-page` comment in reStructuredText). Preview runs the
 native development workflow without editing generated output. Validation checks
 native navigation, configuration, metadata, routes, and conventions; shared
 validation checks professional content, local links, placeholders, and secrets.
 
+Navigation validation must understand the site as the framework builds it,
+not only the two-page scaffold: nested indexes, section pages, autogenerated
+groups, and multi-sidebar objects. When the configuration cannot be read
+statically (a theme, a plugin, a function, an import), return a single
+`navigationUnverifiedIssue` warning from `@doxbrix/doxloop/generator-runtime`
+instead of `unnavigated-page` errors; `contentRelativePages` from the same
+module gives content-relative page paths for comparison.
+
 The format skill must use `$doxloop-authoring`, route all reader pages through
 native navigation, document native components and theme mapping, identify
 generated directories, and require both `doxloop test` and the strict native
@@ -55,5 +105,18 @@ packages declare a compatible core peer dependency and are published after the
 matching core version.
 
 Tests should scaffold, load, validate, install the format skill, build, and
-exercise preview failure behavior. Skill validation checks frontmatter,
-metadata, local reference links, shared-skill delegation, and generator routing.
+exercise preview failure behavior. `src/generator-navigation.test.ts` holds a
+nested-navigation fixture per generator; add one for any adapter you change.
+Skill validation checks frontmatter, metadata, local reference links (fenced
+code is skipped), shared-skill delegation, and generator routing. Every
+generator skill ships `references/authoring.md` covering callouts, tabs, code
+blocks, images, and diagrams in that generator's syntax.
+
+The GitHub Actions workflow in `.github/workflows/ci.yml` runs typecheck, unit
+tests, and skill validation on every pull request, then scaffolds, previews,
+and builds each generator through `scripts/ci-generator-smoke.mjs`, which is
+also runnable locally:
+
+```bash
+pnpm run build && node scripts/ci-generator-smoke.mjs mkdocs
+```

package/docs/generator-selection.md

@@ -1,25 +1,62 @@
 # Select a documentation generator
 
-Choose the generator when initializing a project. Doxbrix is built in; external
-generators require their package in the new project.
+Choose the generator in the setup wizard's **Tools** step, under
+**Documentation generator**. Doxbrix is built in and recommended; choosing
+another generator installs its adapter package into the new documentation
+project before the workspace opens.
 
 Use Doxbrix for native Doxbrix publishing and interactive API endpoint blocks.
 Choose another generator when an existing toolchain, extension ecosystem, or
-static-hosting requirement is more important.
+static-hosting requirement is more important. Native generators may also need
+their own runtime, such as Python for MkDocs and Sphinx, Ruby for Jekyll, or
+the Hugo binary.
 
-`doxloop generator add <name>` installs support; it does not rewrite the project
-or convert existing content. Doxloop currently has no automatic generator
-migration because navigation, frontmatter, components, themes, and route rules
-are not mechanically equivalent.
+## Support tiers and toolchain check
+
+Every generator in the list carries a tier and the toolchain it needs. The
+tier describes what Doxloop's adapter does for that generator today, not how
+capable the framework is:
+
+| Tier | Generators | What Doxloop provides |
+| --- | --- | --- |
+| Full | Doxbrix, Docusaurus, MkDocs Material | Navigation, callouts, tabs, and Mermaid diagrams validated and documented; scaffold, preview, and strict build exercised in CI |
+| Supported | Sphinx, Hugo, VitePress, Starlight | Nested navigation validated wherever the configuration is readable (nested toctrees, section pages and menus, multi-sidebar objects, autogenerated groups); a `navigation-unverified` warning instead of false errors when a theme, plugin, or code builds it; authoring syntax documented |
+| Basic | Nextra, Markdoc, Jekyll, Static HTML | Scaffold, preview, and build work; navigation checks cover the files the scaffold owns; authoring syntax documented |
+
+When you pick a generator in the **Tools** step, Doxloop checks the toolchain
+it needs on this machine: Node.js 22.13 or later with a package manager,
+Python 3.9 or later with `venv`, the `hugo` binary, or Ruby with Bundler. A
+missing tool does not stop the project from being created, since the adapter
+package installs with npm, but preview and the strict build will not run until
+it is installed. **Check again** re-runs the check after you install
+something. **What each generator supports** under the fields lists every
+generator with its tier and requirements.
+
+Validation reports the warning `navigation-unverified` when a generator's
+navigation is produced by something Doxloop cannot read statically: a Hugo
+theme, a MkDocs navigation plugin, a Starlight plugin, a sidebar built by a
+function or imported from another module, a Sphinx `autosummary` toctree, or
+a Jekyll theme that derives navigation from front matter. The native strict
+build, which every publish runs before it uploads, remains the authority in
+those projects.
+
+The active generator is shown under **Settings → Generator**. It is
+deliberately read-only there: navigation, frontmatter, components, themes, and
+route rules are not mechanically equivalent between generators, so Doxloop has
+no in-place conversion.
 
 To migrate:
 
-1. Create a new project with the target generator.
-2. Copy the confirmed documentation brief and source bindings.
-3. Ask `doxloop create` to preserve the agreed coverage while translating
-   content into the target generator's native format.
-4. Compare routes, navigation, metadata, components, and theme behavior.
-5. Run `doxloop test` and the target generator's strict build.
+1. Open the control center from the folder where the new documentation project
+   should live and choose **New documentation project** with the target
+   generator.
+2. Connect the same sources and enter the same documentation brief.
+3. In the guidance step, ask the planner to preserve the agreed coverage while
+   translating content into the target generator's native format.
+4. Review the proposal, comparing routes, navigation, metadata, components,
+   and theme behaviour with the existing site.
+5. Use **Preview docs**, then publish: every publish runs the target
+   generator's strict build first and stops before uploading if it fails.
 6. Redirect changed public URLs before replacing the existing site.
 
 Keep the original project until the new site passes validation and route review.

package/docs/mintlify-import.md

@@ -0,0 +1,71 @@
+# Import Mintlify documentation
+
+Doxloop converts an existing Mintlify project into a new Doxbrix documentation
+workspace. Docusaurus, MkDocs, and other supported generators continue to use
+their native existing-site adoption flow.
+
+## Select and convert
+
+1. Start `doxloop ui`.
+2. In setup, choose **Use existing documentation folder**, then **Mintlify to
+   Doxbrix**. From an open workspace, use the project switcher → **Import
+   existing documentation…**.
+3. Choose **Local folder** and browse to the Mintlify project, or choose
+   **GitHub repository** and enter its URL or `owner/repository`.
+4. Leave the branch blank to use GitHub's default branch. Doxloop finds a single
+   nested Mintlify site automatically; specify a documentation subfolder when
+   the repository contains several sites. Private repositories use local Git
+   credentials, or an access token entered under **Private repository access**.
+5. Choose **Review conversion**. Check the page, asset, navigation-space, and
+   redirect counts and any unsupported constructs or unresolved API references.
+6. Choose a new output folder outside the original repository. The parent folder
+   must already exist. Existing destinations are never overwritten.
+7. Choose **Convert and open Doxbrix project**, then **Open documentation**.
+
+The preview is a frozen conversion snapshot, including assets. Submitting it
+does not rerun conversion against potentially changed source files. An
+inspection expires after 30 minutes; inspect again if it expires.
+
+## Update and publish
+
+Review the converted pages in **Pages** and use **Preview docs**. Direct editing,
+scoped agent changes, review proposals, and undo use the same workflow as any
+other Doxbrix project.
+
+Connect product code or an API specification in **Sources** before asking
+Doxloop to verify product claims or keep documentation synchronized. Conversion
+does not establish factual correctness: every imported page starts unverified,
+and the original Mintlify repository is recorded as migration provenance rather
+than automatically configured as product evidence.
+
+Use **Deploy → Doxbrix** to sign in, run a dry run, and publish. Import itself
+needs neither an agent run nor a Doxbrix account.
+
+## Conversion fidelity and reports
+
+Versioned projects show a sidebar version selector in the Doxbrix reader. Tabs
+and navigation belong to the selected version, and the manifest's `default` or
+`isDefault` entry controls the landing page. Shared pages retain the selected
+version in preview and static exports. The navigation editor also separates
+versions and preserves their metadata when saving changes.
+
+The bundled Doxbrix SDK converter and CLI file helpers handle the native
+manifest, navigation, branding, versions/locales, Markdown/MDX, supported
+components, reusable snippets, images, fonts, media, and API specifications.
+The same OpenAPI materializer used by the CLI expands navigation-generated
+endpoint pages. Remote specifications use Doxloop's bounded public-URL fetcher.
+
+Unsupported constructs and unresolved API sources appear before import and
+require explicit acknowledgement. Review these pages before publishing; custom
+components and source-platform behavior cannot always be represented exactly.
+Custom source CSS and application code are not copied as executable tooling.
+
+The project stores its source location or repository/branch/commit, conversion
+counts, redirects, unsupported constructs, and warnings in
+`.doxloop/mintlify-import.json`. Credentials are not stored there. Redirects are
+also retained in the native manifest and Doxloop's redirect metadata.
+
+The converter source and original regression tests are vendored without changes.
+See `vendor/doxbrix-import/README.md` and its file-hash manifest for provenance
+and refresh instructions. The conversion runtime ships inside the Doxloop
+package and does not require a separate CLI installation.

package/docs/openapi-security.md

@@ -0,0 +1,25 @@
+# Remote OpenAPI safety policy
+
+Doxloop treats remote OpenAPI URLs as untrusted input. Connection tests on the
+**Sources** page, discovery and planning, coverage, and monitoring checks all
+use the same versioned OpenAPI connector and enforce these controls:
+
+- only HTTP and HTTPS URLs without embedded credentials are accepted;
+- localhost, `.local` names, private, loopback, link-local, multicast, and
+  reserved IP ranges are rejected after DNS resolution on every redirect;
+- redirects are handled manually and limited to three;
+- requests time out after 15 seconds and response bodies are limited to 5 MB;
+- JSON, YAML, plain-text, or octet-stream responses are accepted only when the
+  payload parses as a valid OpenAPI 3.x or Swagger 2.0 document; other content
+  types require a `.json`, `.yaml`, or `.yml` URL path;
+- HTTP credentials are never persisted. Private specifications should use a
+  public, credential-free, or deliberately short-lived pre-signed URL, or be
+  uploaded as a file from the **Add source** dialog;
+- ETag and Last-Modified validators are retained in the local, mode-0600 cache.
+  Cached specification bodies stay under `.doxloop/cache/` and are excluded
+  from proposals and deployments.
+
+Structural snapshots hash operations, parameters, request bodies, responses,
+security, examples, schemas, and security schemes. A remote change therefore
+marks only evidence-mapped operations and schemas stale instead of treating the
+entire specification as an opaque file.