@thallylabs/migrate 0.2.12 → 0.2.17

package/README.md

@@ -18,10 +18,18 @@ rendered application before falling back to DOM link inference.
 An `openapi` (or `asyncapi`) reference is resolved wherever Mintlify allows
 it — the top-level `api.openapi`, or on any individual
 tab/anchor/dropdown/product/version, e.g. `navigation.tabs[].openapi` — and
-binds to that exact tab, not only the first one found. Remote `http://` and
+binds to that exact tab, not only the first one found. An `openapi` item inside
+a tab `menu` (string or `{ source, directory }`) becomes its own API tab named
+`<Tab>: <Item>`, so its endpoints appear in the sidebar; when it is the tab's
+only content it binds to the tab itself. The item's `icon` and `hidden` carry
+onto that tab, and a `directory` output folder is not used (a warning says so:
+endpoint pages are generated under `/api/`). An item Thally cannot represent
+produces a warning naming it. Specs that share a file name across folders are
+kept as separate files in `openapi/`. Remote `http://` and
 `https://` specs are not downloaded during migration. The report names each
-remote URL and tells you to add its spec to `public/`, then set the API source
-in `docs.json` to that local path. Missing local specs also produce warnings
+remote URL and tells you to add its spec to a project-root folder such as
+`openapi/`, then set the API source in `docs.json` to that path. Missing local
+specs also produce warnings
 instead of silently dropping that API tab's reference. AsyncAPI has no Thally
 renderer, so it always produces a clear warning naming the spec instead of
 disappearing.
@@ -39,12 +47,20 @@ version first, then pages not listed in navigation, and one warning counts and
 names (the first three of) the pages left out and suggests `--docs-dir` to
 migrate a smaller part of the repository. Scanning itself stops after 100,000
 files with its own warning. Individual assets over 25 MB, or beyond 500 MB in
-total, are not copied and are warned about one by one.
+total, are not copied; one warning per reason gives the count, the first five
+paths, and the next step (copy them into `public/` manually).
 
 Headings written with an explicit `{#custom-id}` become a preceding
 `<a id="custom-id"></a>` anchor and keep their text, so links to that id still
 work and the page still compiles as MDX.
 
+Migrated specs are written to `openapi/`, never `public/`, because the host
+serves `public/` as-is and would publish operations marked `x-excluded` or
+`x-hidden`. Migrating into an existing site does not delete anything: an
+older copy of a spec already in `public/` stays publicly downloadable. The
+migration warns and lists such files, and `thally check` keeps flagging them;
+delete them manually.
+
 Docusaurus sidebars are parsed as bounded, data-only JSON5—never executed—and
 preserve explicit categories, autogenerated slices, document IDs/slugs,
 positions, generated indexes, admonitions, tabs, and `static/` assets where
@@ -229,6 +245,56 @@ YouTube-embed package (e.g. a custom `LiteYouTube` wrapping
 page that imports that package directly, instead of losing the embed to the
 generic unknown-component fallback.
 
+### Access-restricted pages
+
+A gated Mintlify page must never migrate as public. A page is withheld when its
+frontmatter sets a non-empty `groups` (list or string) or `public: false`
+(also `"false"`, `no`, `off`, `0`; empty `groups` restricts nothing), or when it sits
+under a navigation group or tab that sets `groups` (a list of strings) or
+`public: false`. `public: true` alone is not gating. Withheld pages are not
+written to `src/content` and are removed from navigation (emptied groups and
+tabs included), whether or not navigation lists them. The original file is
+written unchanged to `migration-quarantine/<original path>` at the project
+root, which no runtime content reader, search index, or `llms` output walks;
+`create-thally-docs` also adds it to the project's `.gitignore` (appended once,
+existing entries kept) so it is never committed or pushed. A public page that
+imports a withheld page as a component does not inline it (a `gated-page`
+warning names the importer). Invalid frontmatter fails the run rather than
+importing the page. Each page gets a
+`gated-page` warning, plus a summary; `create-thally-docs` prints these last
+under an "ACCESS-RESTRICTED CONTENT" heading and the report records them and a
+`quarantined` count. Links from other pages to a withheld page are left as
+authored and will fail `thally check`; images and other assets used only by
+gated pages are still copied to `public/`. Mintlify's site-wide authentication mode lives in its
+dashboard and cannot be read from a repository, so when any page sets
+`public: true` (which implies the source site required sign-in for everything
+else) one `gated-page` warning says Thally will publish every imported page.
+Restricted-container `groups` and `public: false` are honored fail-closed even
+though Mintlify documents only `"public": true` on a group.
+
+### Site-wide scripts, styles, fonts, and settings
+
+Mintlify serves every `.css` and `.js` file in the content directory on every
+page. Those files are copied to `public/` (tooling config such as
+`*.config.js`, `tailwind.*`, `postcss.*`, plus `dist`/`build`/`coverage`
+output, dot-directories, and `node_modules` are not) under the normal asset
+size limits. Scripts are added to `docs.json` `customScripts` (URL-encoded),
+except modules that pages import as components, which are copied as components
+instead. Thally cannot
+load a stylesheet from `docs.json`, so each `.css` is preserved with a warning
+that it is not applied and likely targets Mintlify's DOM. Fonts named by
+`fonts.source` (top level, `heading`, `body`) that are local `.woff`, `.woff2`,
+`.ttf`, or `.otf` files are copied; Thally's `fonts` setting loads Google Fonts
+by family only, so a warning says the file is not applied. Remote font URLs
+are not downloaded, and unsafe or missing paths are warned about.
+
+Legacy `mint.json` `topbarLinks` and `topbarCtaButton` (including
+`type: github`) map to `navbar.links` and `navbar.primary`; `docs.json`
+`navbar` fields win, with a warning. `appearance.default`, `appearance.strict`
+(`showToggle: false`), and legacy `modeToggle.default` / `isHidden` map to
+`appearance`; invalid values are skipped with a warning. Mintlify sources set
+`markdown.enabled: true`, matching Mintlify's default Markdown mirrors.
+
 ## Branding
 
 A source site's logo, favicon, and theme accent color(s) are extracted where
@@ -245,18 +311,21 @@ dashboard's branding settings instead of leaving the loss silent.
 
 `cloneGitHubRepository` retries a transient network-class failure with
 backoff, and each underlying git process (the clone, and any submodule
-fetch) is killed and treated as a retryable failure if it stalls — a
-generous 10-minute default, configurable via
-`THALLY_MIGRATE_CLONE_TIMEOUT_MS` (milliseconds) for an unusually large
-repository on a slow link. The clone neutralizes the `filter.lfs.*`
+fetch) is killed and treated as a retryable failure if it stalls, meaning
+git prints no progress for 2 minutes (a large repository that is still
+downloading is never cut off; an hour is the absolute ceiling for one git
+process). The allowed silence is configurable via
+`THALLY_MIGRATE_CLONE_TIMEOUT_MS` (milliseconds) for an unusually slow link.
+The clone neutralizes the `filter.lfs.*`
 smudge/clean/process filter driver for that one process only (via
 `GIT_CONFIG_COUNT`/`GIT_CONFIG_KEY_n`/`GIT_CONFIG_VALUE_n`, never the user's
 global git config), so a repository that uses Git LFS still clones even when
 the host has no `git-lfs` binary installed — including a host where
 `filter.lfs.*` is still registered globally from a previous install,
 pointing at a command that no longer exists; an asset that's still a Git
-LFS pointer file afterward (not its real content) is skipped with a warning
-instead of being copied as if it were the real file. Submodules are
+LFS pointer file afterward (not its real content) is skipped, and one
+warning lists those files, instead of copying them as if they were the real
+files. Submodules are
 initialized one at a time, best-effort,
 after a successful clone — a submodule that can't be fetched (private,
 deleted, network trouble) is named in a warning instead of failing the

package/dist/index.d.ts

@@ -113,6 +113,10 @@ interface MigrationDocsConfig {
     feedback?: {
         thumbsRating?: boolean;
     };
+    customScripts?: Array<{
+        src: string;
+        strategy?: 'beforeInteractive' | 'afterInteractive' | 'lazyOnload';
+    }>;
     ai?: {
         chat?: boolean;
         label?: string;
@@ -163,12 +167,16 @@ interface MigrationPage {
     frontmatterError?: string;
 }
 interface MigrationAsset {
-    /** Path below `public`, always normalized and traversal-free. */
+    /** Path below `public` (or below the project root when `projectRelative`), always normalized and traversal-free. */
     path: string;
     content: Uint8Array;
+    /** Write at the project root instead of `public/`, so the host never serves it statically. */
+    projectRelative?: boolean;
 }
 interface MigrationWarning {
-    code: 'collision' | 'invalid-page' | 'missing-page' | 'unsupported-config' | 'limit-reached' | 'fetch-failed' | 'skipped-file';
+    code: 'collision' | 'invalid-page' | 'missing-page' | 'unsupported-config' | 'limit-reached' | 'fetch-failed' | 'skipped-file'
+    /** Access-restricted source page withheld from the published site (or a site-wide auth risk). */
+     | 'gated-page';
     message: string;
     source?: string;
 }
@@ -182,9 +190,16 @@ interface MigrationBundle {
     remoteApiSpecs?: Array<{
         url: string;
         tabLabel?: string;
+        parentTab?: string;
+        icon?: string;
+        hidden?: boolean;
     }>;
     /** Customer-owned component source and registry; paths are repository-relative. */
     componentFiles?: Array<RenderedMigrationFile>;
+    /** Access-restricted source pages, written outside every published path. */
+    quarantinedFiles?: Array<RenderedMigrationFile>;
+    /** Access-restricted pages the file limit dropped: neither published nor saved in quarantinedFiles. */
+    droppedGatedPages?: number;
     docsConfig: MigrationDocsConfig;
     site?: {
         name?: string;
@@ -260,6 +275,8 @@ interface RepositoryMigrationOptions {
     docusaurusSidebarPath?: string;
     /** @internal Static assets are shared across docs-plugin instances. */
     docusaurusSkipAssets?: boolean;
+    /** @internal Test seam: lowers the 5,000-file budget so budget behaviour can be exercised with small fixtures. */
+    maxSourceFiles?: number;
     /** @internal Redirects are global config, read once, not per plugin instance. */
     docusaurusSkipRedirects?: boolean;
 }
@@ -298,9 +315,16 @@ interface MintlifyPageReference {
     navigationId: string;
     locale?: string;
 }
+/** A page listed under an access-restricted navigation container. */
+interface MintlifyGatedReference {
+    ref: string;
+    reason: string;
+}
 interface MintlifyNavigationResult {
     docsConfig: MigrationDocsConfig;
     pageReferences: Array<MintlifyPageReference>;
+    /** Pages under a restricted container; withheld from `docsConfig` and `pageReferences`. */
+    gatedReferences: Array<MintlifyGatedReference>;
     warnings: Array<MigrationWarning>;
 }
 interface MintlifyProjectionOptions {