mdfetch 0.5.2__tar.gz → 0.8.0__tar.gz

mdfetch-0.8.0/.specify/feature.json

@@ -0,0 +1,3 @@
+{
+  "feature_directory": "specs/012-list-platforms"
+}

{mdfetch-0.5.2 → mdfetch-0.8.0}/.specify/memory/changelog.md

@@ -1,3 +1,49 @@
+### mdfetch — CLI: List Supported Platforms — 2026-06-02
+
+**Branch**: `012-list-platforms`
+**Spec**: specs/012-list-platforms
+
+**What was added**:
+- `md-fetch --list-platforms` flag that prints every supported platform domain (alphabetical) to stdout and exits 0 — no URL, no network request.
+- Multi-tenant platforms annotated with their wildcard (e.g. `medium.com (and *.medium.com)`); exact-match platforms shown plain.
+- `URL` argument made optional: required only when `--list-platforms` is absent, preserving the existing `md-fetch <URL>` contract (no breaking change). Missing URL without the flag → Click usage error, exit 2.
+- New subdomain-aware router accessor `supported_platforms() -> list[tuple[str, bool]]` (sorted, registry-derived); existing `supported_domains()` unchanged.
+- `README.md` usage example + `pyproject.toml` version bump `0.7.1` → `0.8.0`.
+
+**New Components**:
+- `src/mdfetch/router.py` — `supported_platforms()` accessor (modified, no new file)
+- `src/mdfetch/cli.py` — `--list-platforms` flag, optional URL (modified, no new file)
+- `tests/unit/test_router.py` — 3 new tests; `tests/unit/test_cli.py` — 5 new tests
+- No integration test (operation is offline)
+
+**Tasks Completed**: 10/10
+
+**Note**: Archived pre-merge at the user's request (deviates from the usual post-merge archival).
+
+---
+
+### mdfetch — Kong Blog Provider — 2026-06-02
+
+**Branch**: `011-konghq-blog-provider`
+**Spec**: specs/011-konghq-blog-provider
+
+**What was added**:
+- `KongExtractor` for `konghq.com/blog/<category>/<slug>` articles. Kong is a Next.js site with hashed CSS-module class names, so selection pins to stable companion classes only.
+- Article discriminator: `<main class="type-article">` (absent on the blog index and category listings). Body = the `<main>` `<section>` richest in `.rich-text-block`.
+- In-body chrome stripped: `.toc-wrap` (TOC sidebar — NOT `[class*=TableOfContents]`, which wraps the whole body), `.component.video`, `.component.more-on-this`, `.order-top`, and trailing non-`intro` `.section-header-block`.
+- `.agent` "agent mode" spans stripped (they inject literal Markdown duplicating styled content; also fixed a `# #` double-heading on the title).
+- Title `<h1>` + publication date prepended (date matched by a month-name regex); author bylines and read time dropped per spec clarification.
+- `README.md` (Supported platforms row + usage example), `pyproject.toml` version bump `0.6.0` → `0.7.0`, and a `CLAUDE.md` gotcha documenting the discriminator and CSS-module fragility.
+
+**New Components**:
+- `src/mdfetch/providers/kong.py` — `KongExtractor`
+- `tests/unit/test_kong_extractor.py` — 12 unit tests
+- `tests/integration/test_kong_integration.py` + 3 snapshots — real-URL tests (3 articles + index/category rejection)
+
+**Tasks Completed**: 26/26
+
+---
+
 ### mdfetch — Homebrew Tap Formula — 2026-05-16
 
 **Branch**: `009-homebrew-tap-formula`

{mdfetch-0.5.2 → mdfetch-0.8.0}/.specify/memory/plan.md

@@ -90,6 +90,16 @@ DZoneExtractor(BaseExtractor) — src/mdfetch/providers/dzone.py
 ├── DOMAINS = frozenset({"dzone.com"})
 ├── _markdownify_kwargs() → overrides to add code_language_callback
 └── clean_html() → locates div.content-html; raises UnsupportedContentTypeError if absent;
+
+KongExtractor(BaseExtractor) — src/mdfetch/providers/kong.py  [011-konghq-blog-provider]
+├── DOMAINS = frozenset({"konghq.com"})
+└── clean_html() → requires <main class="type-article"> (else UnsupportedContentTypeError);
+                   body = the <main> <section> richest in .rich-text-block;
+                   strips in-body chrome (.toc-wrap, .component.video, .component.more-on-this,
+                     .order-top, trailing non-intro .section-header-block) and all .agent spans;
+                   prepends copy(h1) + publication date (class-less hero <div>, month-name regex);
+                   Note: Next.js CSS-module hashes — pin to stable classes only; TOC is .toc-wrap,
+                     NOT [class*=TableOfContents] (that wraps the whole body)
 ```
 
 ### Router / Auto-Discovery
@@ -122,16 +132,17 @@ src/
 └── mdfetch/
     ├── __init__.py          # Public surface: exposes extract(), exception re-exports
     ├── exceptions.py        # MdfetchError hierarchy (6 exception classes)
-    ├── router.py            # @register, _autodiscover_providers(), route()
+    ├── router.py            # @register, _autodiscover_providers(), route(), supported_domains(), supported_platforms() [012-list-platforms]
     ├── base.py              # BaseExtractor ABC + fetch_html() + convert_to_markdown() + extract() template
-    ├── cli.py               # click-based command line interface
+    ├── cli.py               # click CLI; `md-fetch <URL>` + `--list-platforms` (URL optional with flag) [012-list-platforms]
     └── providers/
         ├── __init__.py      # Empty — auto-discovery handles registration
         ├── medium.py        # MediumExtractor
         ├── devto.py         # DevToExtractor [002-devto-provider]
         ├── substack.py      # SubstackExtractor [005-substack-provider]
         ├── thenewstack.py   # TheNewStackExtractor [006-thenewstack-provider]
-        └── dzone.py         # DZoneExtractor
+        ├── dzone.py         # DZoneExtractor
+        └── kong.py          # KongExtractor [011-konghq-blog-provider]
 
 tests/
 ├── unit/
@@ -222,13 +233,15 @@ update-homebrew-tap job (NEW, needs: publish)
 ## Testing Strategy
 
 **Unit tests** (101 tests, offline):
-- Router: domain routing, subdomain suffix matching, duplicate registration, invalid URLs, unsupported platforms
+- Router: domain routing, subdomain suffix matching, duplicate registration, invalid URLs, unsupported platforms; `supported_platforms()` accessor (sorted `(domain, bool)`, covers every `supported_domains()` entry, subdomain flags) [012-list-platforms]
 - MediumExtractor: clean_html, convert_to_markdown, empty content, non-article pages, _parse_freedium (heading remap, missing main-content), fallback on 403/429 (URL construction, exc.url contract, no-sleep on 429), no-fallback on 200, UnsupportedContentTypeError.url on Freedium path [003-medium-freedium-fallback]
 - DevToExtractor: clean_html (title/cover/heading/image preservation, iframe/ltag embed→link, anchor stripping, non-article error), convert_to_markdown (headings/code/lists/images, no raw HTML, empty content error) [002-devto-provider]
 - SubstackExtractor: routing (subdomain + root domain + _no_retry_status_codes assertion), clean_html (body.markup tag return, subscription-widget strip, title prepend, subtitle prepend, prose preservation, iframe→anchor), convert_to_markdown (title heading, no triple blank lines, image syntax, link preservation), paywalled post (non-empty, Subscribe text absent, free preview present), error cases (UnsupportedContentTypeError on no body, EmptyContentError on whitespace body) [005-substack-provider]
 - TheNewStackExtractor: routing (thenewstack.io domain), clean_html (body div return, title prepend, deck-as-paragraph prepend, sponsor note strip, all 3 disclosure variant strips, iframe→anchor, no deck when absent), convert_to_markdown (title heading, deck after title, no triple blank lines, image syntax, link preservation), error cases (UnsupportedContentTypeError on no body, EmptyContentError on whitespace body) [006-thenewstack-provider]
+- KongExtractor: routing (konghq.com domain), clean_html (title+date prepend order, chrome strip — .toc-wrap/.component.video/.component.more-on-this/.order-top/non-intro .section-header-block, .agent affordance strip, body-block preservation, raises without type-article, raises without .rich-text-block), convert_to_markdown (title then date, structure + inline code preserved, chrome/authors excluded, no triple blank lines, EmptyContentError on whitespace body) [011-konghq-blog-provider]
 - Fetch errors: HTTP 404, 503, timeout, connection error, size limit exceeded; `_no_retry_status_codes` immediate-raise + `_no_retry_codes` override [003-medium-freedium-fallback]
 - Silent: no stdout/stderr output, no logging during extraction
+- CLI: `--list-platforms` lists all registered domains (subdomain-annotated), exits 0 with no network and no `extract` call, list takes precedence over a supplied URL, and missing-URL-without-flag exits 2 [012-list-platforms]
 
 **Integration tests** (15 tests, network required):
 - Parametrized over 3 real stn1slv.medium.com articles (including a known paywalled URL that exercises the Freedium fallback when medium.com returns 403) [003-medium-freedium-fallback]
@@ -299,4 +312,11 @@ update-homebrew-tap job (NEW, needs: publish)
 
 ---
 
-*Last Updated: 2026-05-16 | Sources appended: [specs/004-remove-backoff/plan.md], [specs/005-substack-provider/plan.md], [specs/006-thenewstack-provider/plan.md], [specs/009-homebrew-tap-formula/plan.md]*
+### Revision: Archival 2026-06-02
+- Archived **011-konghq-blog-provider**: added the `KongExtractor` architecture block, `kong.py` to the project-structure tree, and its unit/integration test coverage. No new runtime dependencies. [Source: specs/011-konghq-blog-provider]
+- Note: features **007-dzone-provider** (partially present) and **010-boomi-blog-provider** (absent) are not fully archived in this memory plan (gap pre-dating this run).
+
+### Revision: Archival 2026-06-02 (012-list-platforms)
+- Archived **012-list-platforms**: annotated `cli.py` (`--list-platforms`, URL optional) and `router.py` (`supported_platforms()`) in the project-structure tree; added CLI + router-accessor unit-test coverage notes. No new runtime dependencies; no integration test (operation is offline). Reconciled pre-merge at the user's request. [Source: specs/012-list-platforms]
+
+*Last Updated: 2026-06-02 | Sources appended: [specs/004-remove-backoff/plan.md], [specs/005-substack-provider/plan.md], [specs/006-thenewstack-provider/plan.md], [specs/009-homebrew-tap-formula/plan.md], [specs/011-konghq-blog-provider/plan.md], [specs/012-list-platforms/plan.md]*

{mdfetch-0.5.2 → mdfetch-0.8.0}/.specify/memory/spec.md

@@ -223,6 +223,51 @@ A developer runs the integration test suite and all dev.to integration tests pas
 
 ---
 
+### US-019 — Extract a Kong Blog Article to Markdown (P1)
+[Source: specs/011-konghq-blog-provider]
+
+A user passes a public Kong (`konghq.com`) blog article URL and receives the article's title, publication date, and full body as clean Markdown, with all site chrome (navigation, breadcrumbs, table-of-contents, demo CTAs, recommended-posts carousel, author bios, newsletter, footer) removed.
+
+**Acceptance Scenarios**:
+1. Given a public Kong blog article URL, when `extract()` is called, then the result is Markdown whose first line is `# <article title>`, followed by the publication date, then the body content.
+2. Given a Kong article with H2/H3 sections, lists, and inline code, when extracted, then those are preserved as Markdown in document order, and no author byline or read-time text is present.
+
+---
+
+### US-020 — Reject Non-Article Kong URLs (P2)
+[Source: specs/011-konghq-blog-provider]
+
+A user passes a Kong URL that is not a readable article (the blog index or a category listing). The library raises a typed error rather than returning garbage.
+
+**Acceptance Scenarios**:
+1. Given the Kong blog index or a category listing URL, when `extract()` is called, then `UnsupportedContentTypeError` is raised.
+2. Given a Kong article whose body yields no text after stripping, when `extract()` is called, then `EmptyContentError` is raised.
+
+---
+
+### US-021 — List Supported Platforms from the CLI (P1)
+[Source: specs/012-list-platforms]
+
+A user runs the `md-fetch` CLI with a dedicated flag and receives a readable list of every supported platform domain, printed to standard output with a success exit code, without supplying a URL or making any network request.
+
+**Acceptance Scenarios**:
+1. Given the CLI is installed, when the user runs `md-fetch --list-platforms`, then every supported platform domain is printed to standard output and the process exits 0, with no network request made.
+2. Given a new provider has been registered, when the user runs `md-fetch --list-platforms`, then the newly supported domain appears with no change to the list operation itself.
+3. Given neither a URL nor `--list-platforms` is supplied, when `md-fetch` is run, then a usage error is printed and the process exits with code 2 (the existing `md-fetch <URL>` behaviour is unchanged).
+
+---
+
+### US-022 — Distinguish Multi-Tenant Platforms in the Listing (P2)
+[Source: specs/012-list-platforms]
+
+A user listing platforms can tell which platforms also accept per-author subdomains (e.g. Medium, Substack) from those that match an exact domain only.
+
+**Acceptance Scenarios**:
+1. Given a provider that matches subdomains, when the user lists platforms, then the output indicates subdomains of that platform are supported (e.g. `medium.com (and *.medium.com)`).
+2. Given a provider that matches an exact domain only, when the user lists platforms, then the output shows the bare domain with no subdomain indicator.
+
+---
+
 ## Functional Requirements
 
 ### Extraction
@@ -302,6 +347,28 @@ A developer runs the integration test suite and all dev.to integration tests pas
 - **FR-018**: The dev.to provider MUST raise `UnsupportedContentTypeError` when a `dev.to` URL is provided but the page is not an article (e.g., an author profile, a tag listing, or an organisation page). [Source: specs/002-devto-provider]
 - **FR-019**: The dev.to provider MUST replace embedded third-party content (GitHub Gists, CodePen demos, YouTube videos, liquid-tag embeds) with a plain Markdown link to the embedded resource URL. Embeds must not be silently dropped. [Source: specs/002-devto-provider]
 
+### Kong Platform
+- **FR-060**: The library MUST route `konghq.com` blog URLs to the Kong provider using the existing domain-registration mechanism (`@register` decorator + `DOMAINS` frozenset). [Source: specs/011-konghq-blog-provider]
+- **FR-061**: The library MUST extract the main article body from a Kong blog page and return it as clean Markdown. The body is the `<main>` `<section>` richest in `.rich-text-block` blocks. [Source: specs/011-konghq-blog-provider]
+- **FR-062**: The library MUST strip all non-content elements before conversion: navigation, breadcrumbs, topic tags, the table-of-contents sidebar (`.toc-wrap`), demo CTAs, recommended-posts carousel, author byline/bio blocks, the read-time estimate, newsletter signup, and footer. [Source: specs/011-konghq-blog-provider]
+- **FR-063**: The library MUST prepend the article title as a top-level Markdown heading (`# Title`), followed by the publication date when present. Author bylines and read time MUST NOT be included. [Source: specs/011-konghq-blog-provider]
+- **FR-064**: The library MUST preserve structural body content: H2/H3 headings, paragraphs, ordered/unordered lists, inline and block code, blockquotes, hyperlinks, and images inside the body. Images outside the body (hero/banner) MUST be excluded. [Source: specs/011-konghq-blog-provider]
+- **FR-065**: The library MUST raise `UnsupportedContentTypeError` when the page is not a recognisable article — i.e. `<main>` lacks the `type-article` class (blog index, category listing, non-article page). [Source: specs/011-konghq-blog-provider]
+- **FR-066**: The library MUST raise `EmptyContentError` when the Kong article body yields no extractable text after stripping. [Source: specs/011-konghq-blog-provider]
+- **FR-067**: The library MUST collapse runs of three or more consecutive blank lines to a single blank line in the Kong output Markdown. [Source: specs/011-konghq-blog-provider]
+- **FR-068**: The Kong provider MUST depend only on stable, human-authored CSS classes (never Next.js hashed CSS-module suffixes), and MUST strip `.agent` "agent mode" spans that inject literal Markdown duplicating the styled content. [Source: specs/011-konghq-blog-provider]
+
+### CLI — List Platforms
+- **FR-069**: The CLI MUST provide a `--list-platforms` flag on the existing `md-fetch` command that lists all currently supported platform domains. [Source: specs/012-list-platforms]
+- **FR-070**: When `--list-platforms` is supplied, the `URL` argument MUST be optional; without the flag, the existing behaviour (URL required) MUST be unchanged. [Source: specs/012-list-platforms]
+- **FR-071**: The list output MUST be derived from the live provider registry so adding or removing a provider changes the output with no separately maintained list. [Source: specs/012-list-platforms]
+- **FR-072**: The list operation MUST print to standard output and exit with code 0. [Source: specs/012-list-platforms]
+- **FR-073**: The list operation MUST NOT require a URL and MUST NOT perform any network request. [Source: specs/012-list-platforms]
+- **FR-074**: The list output ordering MUST be deterministic (alphabetical by domain). [Source: specs/012-list-platforms]
+- **FR-075**: The list output MUST indicate, for multi-tenant platforms, that subdomains are also supported (`<domain> (and *.<domain>)`), while showing exact-match-only platforms as the bare domain. [Source: specs/012-list-platforms]
+- **FR-076**: The `--list-platforms` flag MUST be documented in the CLI's own `--help` text. [Source: specs/012-list-platforms]
+- **FR-077**: When `--list-platforms` is requested, the CLI MUST NOT attempt extraction in the same invocation, even if a URL is also supplied (the list takes precedence). [Source: specs/012-list-platforms]
+
 ---
 
 ## Key Entities
@@ -369,6 +436,18 @@ A developer runs the integration test suite and all dev.to integration tests pas
 
 A GitHub fine-grained Personal Access Token with `Contents: read+write` on `stn1slv/homebrew-tap` only. Stored as a repository secret in `stn1slv/md-fetch` and consumed exclusively by the `update-homebrew-tap` CI job.
 
+### Kong Blog Post
+[Source: specs/011-konghq-blog-provider]
+| Attribute | Type | Description |
+|-----------|------|-------------|
+| `is-article` | discriminator | `<main>` carries the stable `type-article` class; absence ⇒ non-article |
+| `title` | `str` | Single `<h1>` in the hero section; prepended as `# Title` (with `.agent` `#` affordance stripped) |
+| `date` | `str \| None` | Publication date — a class-less hero `<div>` matched by `^[A-Z][a-z]+ \d{1,2}, \d{4}$`; rendered under the title |
+| `body` | `Tag` | The `<main>` `<section>` richest in `.rich-text-block`; in-body chrome stripped (`.toc-wrap`, `.component.video`, `.component.more-on-this`, `.order-top`, trailing non-`intro` `.section-header-block`) |
+| authors / read time | chrome | Dropped (clarification: keep date only) |
+
+**Validation**: `main.type-article` must exist and a content section must contain ≥1 `.rich-text-block` → else `UnsupportedContentTypeError`. Body must yield non-empty text → else `EmptyContentError`. No paywall — all Kong blog articles are publicly accessible.
+
 ### ExtractionResult (Output)
 | Attribute | Type | Description |
 |-----------|------|-------------|
@@ -389,6 +468,18 @@ MdfetchError (base)
 
 All exceptions carry `message: str` and `url: str | None`. `HTTPStatusError` additionally carries `status_code: int`.
 
+### Supported Platform
+[Source: specs/012-list-platforms]
+
+A read-only projection of one entry in the provider registry, exposed by `router.supported_platforms() -> list[tuple[str, bool]]` (sorted ascending by domain).
+
+| Attribute | Type | Description |
+|-----------|------|-------------|
+| `domain` | `str` | Registered domain (e.g. `medium.com`, `dev.to`) |
+| `matches_subdomains` | `bool` | Whether subdomains of `domain` also route to this provider (the provider's `MATCH_SUBDOMAINS` flag) |
+
+**Validation**: One tuple per registered domain — no missing, no extras; domains unique. This is the subdomain-aware superset of `supported_domains() -> frozenset[str]`, which is retained unchanged.
+
 ---
 
 ## Call Lifecycle / State Transitions
@@ -450,6 +541,11 @@ caller provides URL string
 - **Concurrent releases**: Addressed by `concurrency: group=homebrew-tap-update, cancel-in-progress=false` — runs are serialized so the second release waits for the first tap-update to complete. [Source: specs/009-homebrew-tap-formula]
 - **brew test fails after install**: `brew test md-fetch` exits non-zero when a transitive dependency is missing or `md-fetch --version` fails — installation validation fails. [Source: specs/009-homebrew-tap-formula]
 - **Formula structure changed (sed no-op)**: If the formula is manually edited and the url/sha256 line format changes, `sed` produces no diff and `git commit` finds nothing staged → exits non-zero → CI job fails. [Source: specs/009-homebrew-tap-formula]
+- **Kong non-article pages**: The blog index (`/blog`) and category listings (`/blog/product-releases`) render `<main>` without the `type-article` class (and zero `.rich-text-block`) → `UnsupportedContentTypeError`. [Source: specs/011-konghq-blog-provider]
+- **Kong Next.js CSS-module hashes**: Per-component class names (e.g. `Section_section__Grz_Y`) change between builds; selection pins to stable companion classes only. The TOC sidebar is `.toc-wrap` — NOT `[class*=TableOfContents]`, which wraps the entire body and would delete the article. [Source: specs/011-konghq-blog-provider]
+- **Kong "agent mode" duplicates**: `<span class="agent">` elements inject literal Markdown (`**`, `- `, `# `) duplicating styled content (and cause a `# #` double-heading on the title); all `.agent` spans are decomposed before conversion. [Source: specs/011-konghq-blog-provider]
+- **CLI list + output/fetch options**: When `--list-platforms` is combined with `-o/--output`, `--retries`, or `--force`, the list is printed to stdout and those options are ignored — the list operation returns before any extraction. [Source: specs/012-list-platforms]
+- **CLI missing URL**: Running `md-fetch` with neither a URL nor `--list-platforms` raises a Click usage error and exits with code 2. [Source: specs/012-list-platforms]
 
 ---
 
@@ -486,6 +582,19 @@ caller provides URL string
 - **SC-034**: Formula update failures produce a visible CI job failure on every failed attempt, with zero silent failures. [Source: specs/009-homebrew-tap-formula]
 - **SC-035**: The README install section includes the Homebrew installation command, enabling users to discover and use it without prior knowledge of the project's Python packaging. [Source: specs/009-homebrew-tap-formula]
 
+- **SC-036**: Each of the three reference Kong articles returns Markdown containing the full title and all body section headings and paragraph text, with zero non-content elements. [Source: specs/011-konghq-blog-provider]
+- **SC-037**: Extraction of a Kong article completes within the base class 30-second fetch timeout on a stable connection. [Source: specs/011-konghq-blog-provider]
+- **SC-038**: A non-article Kong URL (the blog index or a category listing) raises `UnsupportedContentTypeError` within the normal fetch timeout. [Source: specs/011-konghq-blog-provider]
+- **SC-039**: The extracted Markdown for any Kong article contains no consecutive blank-line runs of three or more lines. [Source: specs/011-konghq-blog-provider]
+- **SC-040**: The Kong provider is exercised by integration tests using real network requests against the reference URLs, matching the pattern established by existing providers. [Source: specs/011-konghq-blog-provider]
+- **SC-041**: For a Kong article with a visible publication date, the extracted Markdown contains the date directly beneath the title and contains no author byline or read-time text. [Source: specs/011-konghq-blog-provider]
+
+- **SC-042**: A user can list every supported platform with a single `md-fetch --list-platforms` invocation and no URL, completing in well under one second with no network access. [Source: specs/012-list-platforms]
+- **SC-043**: The listed domains exactly match the set of domains registered to providers — no missing entries and no extras. [Source: specs/012-list-platforms]
+- **SC-044**: After a new provider is added, its domain appears in the list output with no edit to the list operation's code. [Source: specs/012-list-platforms]
+- **SC-045**: Multi-tenant platforms (those accepting subdomains) are visually distinguishable from exact-match platforms in the output. [Source: specs/012-list-platforms]
+- **SC-046**: The list operation is exercised by at least one automated unit test that asserts the output contains the known supported domains and exits successfully. [Source: specs/012-list-platforms]
+
 ---
 
 ## Assumptions
@@ -513,4 +622,11 @@ caller provides URL string
 
 ---
 
-*Last Updated: 2026-05-16 | Sources appended: [specs/004-remove-backoff/spec.md], [specs/005-substack-provider/spec.md], [specs/006-thenewstack-provider/spec.md], [specs/009-homebrew-tap-formula/spec.md]*
+### Revision: Archival 2026-06-02
+- Archived **011-konghq-blog-provider**: added US-019/US-020, FR-060–FR-068 (Kong Platform), the Kong Blog Post entity, Kong edge cases, and SC-036–SC-041. [Source: specs/011-konghq-blog-provider]
+- Note: features **007-dzone-provider** and **010-boomi-blog-provider** are not present in this memory spec (un-archived gap pre-dating this run).
+
+### Revision: Archival 2026-06-02 (012-list-platforms)
+- Archived **012-list-platforms**: added US-021/US-022, FR-069–FR-077 (CLI — List Platforms), the Supported Platform entity, two CLI edge cases, and SC-042–SC-046. Reconciled pre-merge at the user's request (deviates from the usual post-merge archival). [Source: specs/012-list-platforms]
+
+*Last Updated: 2026-06-02 | Sources appended: [specs/004-remove-backoff/spec.md], [specs/005-substack-provider/spec.md], [specs/006-thenewstack-provider/spec.md], [specs/009-homebrew-tap-formula/spec.md], [specs/011-konghq-blog-provider/spec.md]*

{mdfetch-0.5.2 → mdfetch-0.8.0}/CLAUDE.md

@@ -17,11 +17,13 @@ src/mdfetch/
     ├── devto.py      # DevToExtractor (dev.to)
     ├── substack.py   # SubstackExtractor (substack.com + *.substack.com)
     ├── thenewstack.py  # TheNewStackExtractor (thenewstack.io)
-    └── dzone.py      # DZoneExtractor (dzone.com)
+    ├── dzone.py      # DZoneExtractor (dzone.com)
+    ├── boomi.py      # BoomiExtractor (boomi.com/blog)
+    └── kong.py       # KongExtractor (konghq.com/blog)
 
 tests/
 ├── unit/             # pytest unit tests (no network)
-└── integration/      # real network tests (Medium + dev.to + Substack + TheNewStack URLs + snapshots)
+└── integration/      # real network tests (Medium + dev.to + Substack + TheNewStack + DZone + Boomi + Kong URLs + snapshots)
 
 .github/workflows/
 ├── ci.yml            # lint + unit tests on push/PR (Python 3.12–3.14)
@@ -40,6 +42,8 @@ Makefile              # setup / test / lint / format / build / upgrade-deps / cl
 - **No logging**: all failures communicated via typed exceptions only (FR-013)
 - **Type hints**: strict (`mypy src/` must pass with zero errors)
 - **Linter/formatter**: `ruff` (`make lint` / `make format`)
+- **Version bump**: every user-facing change MUST bump `version` in `pyproject.toml` (SemVer — new provider/feature = minor, bug fix = patch). Adding a provider follows the dev.to/Substack/Boomi minor-bump precedent. Don't forget this — it gates the PyPI release (`publish.yml`) and the Homebrew tap auto-update.
+- **README sync**: any change to supported platforms or public usage MUST update `README.md` (add the provider to the Supported platforms table + a usage example). Keep the `## Project structure` tree above and the integration-test description in this file in sync too.
 
 ## Common commands
 
@@ -76,4 +80,19 @@ make typecheck    # type check
 **Issue:** `brew audit --strict --new Formula/md-fetch.rb` fails with missing system library declarations when `lxml` is a resource block.
 **Root Cause:** `lxml` requires `libxml2` and `libxslt`, which are macOS system libraries. Homebrew requires these to be declared explicitly via `uses_from_macos`.
 **Prevention Rule:** Any Homebrew formula that includes `lxml` as a resource MUST declare `uses_from_macos "libxml2"` and `uses_from_macos "libxslt"` after the `depends_on` lines. Discovered via `brew audit --strict --new` during implementation.
+
+### ⚠️ boomi.com article detection relies on `div.post-content`, NOT `section.wysiwyg-section`
+**Issue:** The Boomi blog index (`/blog/`) renders a `section.wysiwyg-section` intro but NO `div.post-content`. Selecting `wysiwyg-section` as the body container would fail to raise `UnsupportedContentTypeError` for the index and other non-article pages.
+**Root Cause:** Confirmed via live DOM inspection: only article pages render `div.post-content` (containing `section.wysiwyg-section` + a `div.blog-nav` prev/next block); the index renders `wysiwyg-section` only.
+**Prevention Rule:** `BoomiExtractor.clean_html()` MUST select `div.post-content` as the body container (its presence is the article discriminator) and strip the inner `div.blog-nav`. Do not switch to `wysiwyg-section`. The title `<h1>` lives in the page hero outside the body and is prepended.
+
+### ⚠️ konghq.com is a Next.js site — pin to stable classes, NOT hashed CSS-module suffixes
+**Issue:** Kong's per-component class names are Next.js CSS-module build hashes (e.g. `Section_section__Grz_Y`, `Article_toc__LOyCI`) that change between site builds. Selecting on a `__xxxxx` suffix would silently break on the next Kong deploy.
+**Root Cause:** Confirmed via live DOM inspection: every block carries a hashed module class plus a stable, human-authored companion class. Only the companion classes are durable.
+**Prevention Rule:** `KongExtractor.clean_html()` MUST use only stable classes: `main.type-article` is the **article discriminator** (absent on the blog index and category listings, which also have 0 `.rich-text-block`); the body is the `<section>` richest in `.rich-text-block`; strip in-body chrome via `.component.video`, `.component.more-on-this`, `.toc-wrap`, `.order-top`, and `.section-header-block:not(.intro)`. NOTE: `.toc-wrap` (not `[class*=TableOfContents]`) is the TOC sidebar — the `TableOfContents` component WRAPS the whole body, so matching it would delete the article. Also strip `.agent` spans everywhere: they are "agent mode" affordances that inject literal Markdown (`**`, `- `, `# `) duplicating the styled content. Title `<h1>` + the publication date (a class-less hero `<div>` matched by a month-name regex) are prepended; author bylines and read time are dropped.
+
+### ⚠️ CLI `url` argument is `required=False` to support `--list-platforms`
+**Issue:** `md-fetch --list-platforms` takes no URL, so the Click `url` argument is declared `required=False`. A future refactor that restores `required=True` (or drops the manual `url is None` check) would break `--list-platforms` and change the missing-URL exit code.
+**Root Cause:** Click cannot express "required unless another flag is set"; requiredness is enforced manually in `main()` (`raise click.UsageError(...)` → exit 2) after the early `--list-platforms` return.
+**Prevention Rule:** Keep `@click.argument("url", required=False)` and the manual `if url is None: raise click.UsageError(...)` guard. The list branch MUST `return` before any extraction so `--list-platforms` never makes a network call (FR-077). Use `router.supported_platforms()` (subdomain-aware) for the listing — NOT `supported_domains()`. [012-list-platforms]
 <!-- SPECKIT END -->

{mdfetch-0.5.2 → mdfetch-0.8.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mdfetch
-Version: 0.5.2
+Version: 0.8.0
 Summary: Extract article content from web platforms and return it as clean Markdown.
 Project-URL: Homepage, https://github.com/stn1slv/md-fetch
 Project-URL: Source, https://github.com/stn1slv/md-fetch
@@ -61,6 +61,9 @@ md-fetch https://medium.com/example/article
 
 # Fetch and save Markdown to a file
 md-fetch https://dev.to/example/article --output article.md
+
+# List all supported platforms (no URL or network needed)
+md-fetch --list-platforms
 ```
 
 ## Python Usage
@@ -74,6 +77,8 @@ markdown = extract("https://dev.to/username/article-slug")
 markdown = extract("https://example.substack.com/p/article-slug")
 markdown = extract("https://thenewstack.io/article-slug")
 markdown = extract("https://dzone.com/articles/article-slug")
+markdown = extract("https://boomi.com/blog/article-slug")
+markdown = extract("https://konghq.com/blog/category/article-slug")
 print(markdown)
 ```
 
@@ -117,6 +122,8 @@ except EmptyContentError as e:
 | Substack | `substack.com`, `*.substack.com` |
 | The New Stack | `thenewstack.io` |
 | DZone | `dzone.com` |
+| Boomi | `boomi.com` |
+| Kong | `konghq.com` |
 
 ## Development
 

{mdfetch-0.5.2 → mdfetch-0.8.0}/README.md

@@ -28,6 +28,9 @@ md-fetch https://medium.com/example/article
 
 # Fetch and save Markdown to a file
 md-fetch https://dev.to/example/article --output article.md
+
+# List all supported platforms (no URL or network needed)
+md-fetch --list-platforms
 ```
 
 ## Python Usage
@@ -41,6 +44,8 @@ markdown = extract("https://dev.to/username/article-slug")
 markdown = extract("https://example.substack.com/p/article-slug")
 markdown = extract("https://thenewstack.io/article-slug")
 markdown = extract("https://dzone.com/articles/article-slug")
+markdown = extract("https://boomi.com/blog/article-slug")
+markdown = extract("https://konghq.com/blog/category/article-slug")
 print(markdown)
 ```
 
@@ -84,6 +89,8 @@ except EmptyContentError as e:
 | Substack | `substack.com`, `*.substack.com` |
 | The New Stack | `thenewstack.io` |
 | DZone | `dzone.com` |
+| Boomi | `boomi.com` |
+| Kong | `konghq.com` |
 
 ## Development
 

{mdfetch-0.5.2 → mdfetch-0.8.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "mdfetch"
-version = "0.5.2"
+version = "0.8.0"
 description = "Extract article content from web platforms and return it as clean Markdown."
 readme = "README.md"
 license = { text = "MIT" }

mdfetch-0.8.0/specs/010-boomi-blog-provider/checklists/requirements.md

@@ -0,0 +1,37 @@
+# Specification Quality Checklist: Boomi Blog Provider
+
+**Purpose**: Validate specification completeness and quality before proceeding to planning
+**Created**: 2026-06-02
+**Feature**: [spec.md](../spec.md)
+
+## Content Quality
+
+- [x] No implementation details (languages, frameworks, APIs)
+- [x] Focused on user value and business needs
+- [x] Written for non-technical stakeholders
+- [x] All mandatory sections completed
+
+## Requirement Completeness
+
+- [x] No [NEEDS CLARIFICATION] markers remain
+- [x] Requirements are testable and unambiguous
+- [x] Success criteria are measurable
+- [x] Success criteria are technology-agnostic (no implementation details)
+- [x] All acceptance scenarios are defined
+- [x] Edge cases are identified
+- [x] Scope is clearly bounded
+- [x] Dependencies and assumptions identified
+
+## Feature Readiness
+
+- [x] All functional requirements have clear acceptance criteria
+- [x] User scenarios cover primary flows
+- [x] Feature meets measurable outcomes defined in Success Criteria
+- [x] No implementation details leak into specification
+
+## Notes
+
+- Items marked incomplete require spec updates before `/speckit-clarify` or `/speckit-plan`
+- Note: FR-001 and the Key Entities section reference the existing provider/`@register` mechanism. This is an
+  intentional, established convention in this repository's spec template (see prior provider specs 002–007),
+  describing the routing contract rather than implementation internals. Retained for consistency.

mdfetch-0.8.0/specs/010-boomi-blog-provider/contracts/extractor-contract.md

@@ -0,0 +1,57 @@
+# Provider Contract: BoomiExtractor
+
+This is a library; its external contracts are (1) the public `extract()` API and
+(2) the `BaseExtractor` subclass contract the new provider must satisfy.
+
+## 1. Public API (unchanged)
+
+```python
+from mdfetch import extract
+
+markdown: str = extract("https://boomi.com/blog/<slug>/", retries=3, retry_delay=2.0)
+```
+
+- **Input**: a `boomi.com` blog article URL (`str`).
+- **Output**: clean Markdown (`str`), title-first.
+- **Raises**: `UnsupportedPlatformError` (domain not registered — N/A once Boomi is registered),
+  `UnsupportedContentTypeError`, `EmptyContentError`, `HTTPStatusError`, `FetchError`
+  (all from `mdfetch.exceptions`). No new exception types.
+
+## 2. Subclass contract
+
+```python
+@register
+class BoomiExtractor(BaseExtractor):
+    DOMAINS: frozenset[str] = frozenset({"boomi.com"})
+    # MATCH_SUBDOMAINS stays False (default)
+
+    def clean_html(self, soup: BeautifulSoup) -> Tag: ...
+```
+
+### `clean_html(soup)` obligations
+
+| # | Obligation |
+|---|------------|
+| C1 | Return the `div.post-content` `Tag` with chrome stripped and the title prepended. |
+| C2 | Raise `UnsupportedContentTypeError` when `div.post-content` is absent. |
+| C3 | Decompose every `div.blog-nav` descendant before returning. |
+| C4 | Prepend a copy of the page `<h1>` as the first child of the returned body. |
+| C5 | MUST NOT modify the base class or shared utilities. |
+| C6 | MUST NOT add image-specific logic (body images flow through default conversion). |
+
+`extract()` and `convert_to_markdown()` are inherited unchanged; the latter enforces the
+ATX-heading, blank-line-collapsing, and `EmptyContentError` behavior.
+
+## 3. Acceptance contract (maps to spec SC-00x)
+
+| Check | Maps to |
+|-------|---------|
+| Each reference article → Markdown begins with `# <title>` and contains body section headings + paragraph text; no chrome | SC-001, FR-002/003/004/005 |
+| `extract("https://boomi.com/blog/")` (index) raises `UnsupportedContentTypeError` | SC-003, FR-006, FR-009 |
+| Output has no 3+ consecutive blank lines | SC-004, FR-008 |
+| ≥1 integration test hits a real reference URL with snapshot containment | SC-005 |
+
+## 4. Routing contract
+
+- After `@register`, `route("https://boomi.com/blog/...")` resolves to `BoomiExtractor`.
+- `test_router.py` unsupported-domain fixture (`wordpress.com`) remains valid — no change needed.

mdfetch-0.8.0/specs/010-boomi-blog-provider/data-model.md

@@ -0,0 +1,45 @@
+# Phase 1 Data Model: Boomi Blog Provider
+
+This library is stateless; there are no persisted entities. The "data model" describes
+the DOM structures the extractor reads and the in-memory shapes it produces.
+
+## Entity: Boomi Blog Post (input DOM)
+
+A single article page at `https://boomi.com/blog/<slug>/`.
+
+| Field | Source selector | Notes |
+|-------|-----------------|-------|
+| title | `h1` (within `section.post-detail-hero`) | Exactly one per page; outside the body container |
+| body | `div.post-content` | Body container; presence = "is an article" |
+| content | `section.wysiwyg-section.bullet-styled` | Real article content (child of body) |
+| nav (chrome) | `div.blog-nav` | Prev/next post links (child of body) — stripped |
+| images | `img` inside `div.post-content` | Preserved (body-only, per clarification) |
+| blockquotes | `blockquote` inside content | Preserved natively |
+| subtitle/deck | — | Not present on any reference article |
+
+**Validation rules**:
+- `div.post-content` MUST exist → else `UnsupportedContentTypeError`.
+- After stripping and conversion, Markdown MUST be non-empty → else `EmptyContentError`.
+
+## Entity: Extraction Result (output)
+
+A single Markdown `str` (the public `extract()` return value):
+- Line 1: `# <title>` (top-level ATX heading).
+- Followed by the converted body: headings, paragraphs, lists, blockquotes, links, and
+  any in-body images, in document order.
+- No runs of 3+ consecutive blank lines (collapsed by base `convert_to_markdown`).
+- Contains no site chrome (nav, language selector, CTAs, TOC, share buttons, blog-nav,
+  sidebar promos, footer).
+
+## DOM Element Taxonomy (selector → action)
+
+| Selector | Action | Reason |
+|----------|--------|--------|
+| `div.post-content` | **keep** (body root) | Article body; absence ⇒ non-article |
+| `div.blog-nav` (inside body) | **strip** (`decompose`) | Prev/next post chrome |
+| `h1` (hero) | **prepend** (copy into body) | Article title heading |
+| everything outside `div.post-content` | **excluded** implicitly | Nav, TOC, share, sidebar promos, footer, hero image |
+
+## State Transitions
+
+None. Each `extract(url)` call is independent: `fetch → parse → clean → convert → return | raise`.