fontlab-www-toolkit 1.0.4.dev3__tar.gz → 1.0.5.dev3__tar.gz

{fontlab_www_toolkit-1.0.4.dev3 → fontlab_www_toolkit-1.0.5.dev3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: fontlab-www-toolkit
-Version: 1.0.4.dev3
+Version: 1.0.5.dev3
 Summary: Shared ProperDocs + MaterialX site builder and deploy helpers for FontLab web properties.
 Project-URL: Homepage, https://github.com/Fontlab/fontlab-www-toolkit
 Project-URL: Issues, https://github.com/Fontlab/fontlab-www-toolkit/issues
@@ -26,7 +26,6 @@ Requires-Dist: mkdocs>=1.6.1
 Requires-Dist: properdocs>=1.6.7
 Description-Content-Type: text/markdown
 
-<!-- this_file: README.md -->
 
 # fontlab-www-toolkit
 
@@ -56,8 +55,8 @@ Commands:
 
 | Command | Effect |
 |---|---|
-| `build [--skip_webflow]` | Pull Webflow stubs, build with MkDocs/ProperDocs, overlay `wf_cache/` + `static_docs/`, publish to `public/`. |
-| `pull-webflow` | Refresh `wf_cache/` only. |
+| `build [--skip_webflow] [--update_stubs]` | Pull Webflow stubs, build with MkDocs/ProperDocs, overlay `wf_cache/` + `static_docs/`, publish to `public/`. |
+| `pull-webflow [--update_stubs]` | Refresh `wf_cache/` only. |
 | `convert-old` | Regenerate OLD pages from `src_docs/old-pages.yml`. |
 | `clean` | Delete `build_docs/` and `public/`. |
 | `setup [--venv PATH] [--clear]` | Create / refresh a uv venv for the admin pipeline. |
@@ -88,6 +87,24 @@ webflow-import-url: https://example.webflow.io/page
 ---
 ```
 
+### Refreshing stub bodies (`--update_stubs`)
+
+Normally only `wf_cache/` is refreshed; the stub Markdown body stays as a
+placeholder (the cached HTML overlays it at build time). Pass `--update_stubs`
+to `pull-webflow` or `build` to also rewrite each stub's **body** from the
+freshly cached HTML, while preserving the stub's **frontmatter** verbatim:
+
+```bash
+fontlab-www-toolkit pull-webflow --update_stubs
+```
+
+For each stub it strips non-prose noise (`<script>`/`<style>`/`<noscript>` and
+hidden Webflow *Windflow* plugin metadata), serves the cleaned cached HTML over
+a loopback HTTP server, and runs [`url22md`](https://github.com/twardoch/url22md)
+to extract Markdown. This requires the `url22md` CLI on `PATH` (or set
+`url22md_bin` in config); it is an external runtime tool, not a packaged
+dependency.
+
 ## Configuration
 
 You can customize the builder's behavior using a JSON configuration file (by default `fontlab-www-toolkit.json` in the root directory, or passed via `--config` CLI option). Alternatively, individual pages can specify page-specific overrides inside the input HTML:
@@ -115,6 +132,10 @@ The following settings can be overridden in the root configuration object:
 * **`old_pages_config`** (string): Path to the legacy pages mapping file. Defaults to `"src_docs/old-pages.yml"`.
 * **`mkdocs_command`** (string | array): The custom build command for MkDocs/ProperDocs. Can include the `{config_file}` placeholder.
 * **`user_agent`** (string): Custom User-Agent header used when pulling Webflow pages. Defaults to `"fontlab_www_toolkit"`.
+* **`split_google_fonts`** (bool): When true (default), split a multi-family `fonts.googleapis.com/css2?family=A&family=B&…` link into one `?family=X&display=swap` link per family. This keeps web fonts loading even if a downstream step truncates the served URL at the first `&` (otherwise only the first family loads and the rest fall back to a system font).
+* **`url22md_bin`** (string): Path to the `url22md` executable used by `--update_stubs`. Defaults to whatever is found on `PATH`.
+* **`url22md_tool`** (int | null): Forces a specific `url22md` extraction engine (`1`=trafilatura, `3`=readability, …). Defaults to `1` for offline, deterministic extraction; set to `null` to let url22md run its full fallback chain (may reach cloud tools).
+* **`url22md_timeout`** (int): Per-page extraction timeout in seconds. Defaults to `60`.
 
 ### Cloudinary Options
 

{fontlab_www_toolkit-1.0.4.dev3 → fontlab_www_toolkit-1.0.5.dev3}/README.md

@@ -1,4 +1,3 @@
-<!-- this_file: README.md -->
 
 # fontlab-www-toolkit
 
@@ -28,8 +27,8 @@ Commands:
 
 | Command | Effect |
 |---|---|
-| `build [--skip_webflow]` | Pull Webflow stubs, build with MkDocs/ProperDocs, overlay `wf_cache/` + `static_docs/`, publish to `public/`. |
-| `pull-webflow` | Refresh `wf_cache/` only. |
+| `build [--skip_webflow] [--update_stubs]` | Pull Webflow stubs, build with MkDocs/ProperDocs, overlay `wf_cache/` + `static_docs/`, publish to `public/`. |
+| `pull-webflow [--update_stubs]` | Refresh `wf_cache/` only. |
 | `convert-old` | Regenerate OLD pages from `src_docs/old-pages.yml`. |
 | `clean` | Delete `build_docs/` and `public/`. |
 | `setup [--venv PATH] [--clear]` | Create / refresh a uv venv for the admin pipeline. |
@@ -60,6 +59,24 @@ webflow-import-url: https://example.webflow.io/page
 ---
 ```
 
+### Refreshing stub bodies (`--update_stubs`)
+
+Normally only `wf_cache/` is refreshed; the stub Markdown body stays as a
+placeholder (the cached HTML overlays it at build time). Pass `--update_stubs`
+to `pull-webflow` or `build` to also rewrite each stub's **body** from the
+freshly cached HTML, while preserving the stub's **frontmatter** verbatim:
+
+```bash
+fontlab-www-toolkit pull-webflow --update_stubs
+```
+
+For each stub it strips non-prose noise (`<script>`/`<style>`/`<noscript>` and
+hidden Webflow *Windflow* plugin metadata), serves the cleaned cached HTML over
+a loopback HTTP server, and runs [`url22md`](https://github.com/twardoch/url22md)
+to extract Markdown. This requires the `url22md` CLI on `PATH` (or set
+`url22md_bin` in config); it is an external runtime tool, not a packaged
+dependency.
+
 ## Configuration
 
 You can customize the builder's behavior using a JSON configuration file (by default `fontlab-www-toolkit.json` in the root directory, or passed via `--config` CLI option). Alternatively, individual pages can specify page-specific overrides inside the input HTML:
@@ -87,6 +104,10 @@ The following settings can be overridden in the root configuration object:
 * **`old_pages_config`** (string): Path to the legacy pages mapping file. Defaults to `"src_docs/old-pages.yml"`.
 * **`mkdocs_command`** (string | array): The custom build command for MkDocs/ProperDocs. Can include the `{config_file}` placeholder.
 * **`user_agent`** (string): Custom User-Agent header used when pulling Webflow pages. Defaults to `"fontlab_www_toolkit"`.
+* **`split_google_fonts`** (bool): When true (default), split a multi-family `fonts.googleapis.com/css2?family=A&family=B&…` link into one `?family=X&display=swap` link per family. This keeps web fonts loading even if a downstream step truncates the served URL at the first `&` (otherwise only the first family loads and the rest fall back to a system font).
+* **`url22md_bin`** (string): Path to the `url22md` executable used by `--update_stubs`. Defaults to whatever is found on `PATH`.
+* **`url22md_tool`** (int | null): Forces a specific `url22md` extraction engine (`1`=trafilatura, `3`=readability, …). Defaults to `1` for offline, deterministic extraction; set to `null` to let url22md run its full fallback chain (may reach cloud tools).
+* **`url22md_timeout`** (int): Per-page extraction timeout in seconds. Defaults to `60`.
 
 ### Cloudinary Options
 

{fontlab_www_toolkit-1.0.4.dev3 → fontlab_www_toolkit-1.0.5.dev3}/src/fontlab_www_toolkit/__main__.py

@@ -22,27 +22,42 @@ from fontlab_www_toolkit.builder import BuildPaths, SiteBuilder, setup_environme
 class Cli:
     """Build, pull-webflow, convert-old, clean, or setup a FontLab site repo."""
 
-    def build(self, root: str | None = None, skip_webflow: bool = False, config: str | None = None) -> None:
+    def build(
+        self,
+        root: str | None = None,
+        skip_webflow: bool = False,
+        update_stubs: bool = False,
+        config: str | None = None,
+    ) -> None:
         """Full build: pull Webflow (unless skipped), build, overlay, publish.
 
         Args:
             root: site repo root; defaults to the current working directory.
             skip_webflow: if true, do not refresh ``wf_cache/`` before building.
+            update_stubs: if true, rewrite each Webflow stub's body from the
+                cached HTML (via url22md), preserving the original frontmatter.
             config: path to the JSON configuration file.
         """
         builder, paths = _builder(root, config=config)
-        builder.build(pull_webflow=not skip_webflow)
+        builder.build(pull_webflow=not skip_webflow, update_stubs=update_stubs)
         print(f"Published {paths.public}")
 
-    def pull_webflow(self, root: str | None = None, config: str | None = None) -> None:
+    def pull_webflow(
+        self,
+        root: str | None = None,
+        update_stubs: bool = False,
+        config: str | None = None,
+    ) -> None:
         """Refresh ``wf_cache/`` only.
 
         Args:
             root: site repo root; defaults to the current working directory.
+            update_stubs: if true, rewrite each Webflow stub's body from the
+                cached HTML (via url22md), preserving the original frontmatter.
             config: path to the JSON configuration file.
         """
         builder, paths = _builder(root, config=config)
-        for path in builder.pull_webflow():
+        for path in builder.pull_webflow(update_stubs=update_stubs):
             print(path.relative_to(paths.root))
 
     def convert_old(self, root: str | None = None, config: str | None = None) -> None:
@@ -79,6 +94,7 @@ class Cli:
     def version(self) -> None:
         """Print the package version and exit."""
         from fontlab_www_toolkit import __version__
+
         print(__version__)
 
 

{fontlab_www_toolkit-1.0.4.dev3 → fontlab_www_toolkit-1.0.5.dev3}/src/fontlab_www_toolkit/__version__.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '1.0.4.dev3'
-__version_tuple__ = version_tuple = (1, 0, 4, 'dev3')
+__version__ = version = '1.0.5.dev3'
+__version_tuple__ = version_tuple = (1, 0, 5, 'dev3')
 
 __commit_id__ = commit_id = None