any2md 1.0.5__tar.gz → 1.0.7__tar.gz

{any2md-1.0.5 → any2md-1.0.7}/CHANGELOG.md

@@ -4,6 +4,119 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
+## [1.0.7] — 2026-04-29
+
+Branding/rebrand release. The frontmatter contract that any2md emits is
+now described as "SAGE-compatible" (Security Analysis and Guidance
+Exchange) — the upstream specification was renamed from SSRM (Structured
+Security Reasoning Markdown) to SAGE. No functional or schema changes:
+field names, ordering, controlled vocabularies, and `content_hash`
+derivation are byte-identical to v1.0.6. Existing v1.0.6 outputs remain
+valid SAGE-compatible documents without modification.
+
+### Changed
+
+- README, CHANGELOG, and all of `docs/` updated to reference SAGE /
+  Security Analysis and Guidance Exchange.
+- `--auto-id` CLI help text now reads "SAGE-conformant document_id"
+  (previously "SSRM-conformant"). Behavior unchanged.
+- Module/function docstrings in `any2md/frontmatter.py`,
+  `any2md/converters/txt.py`, and `any2md/pipeline/cleanup.py` updated
+  to reference SAGE §3 / §5.1 (same section numbers as the upstream
+  SSRM v1.0-RC1 spec).
+- Two test functions renamed for naming consistency
+  (`test_txt_end_to_end_writes_sage_compat_output`,
+  `test_compose_emits_required_sage_fields`). Pytest auto-discovery is
+  unaffected; assertions unchanged.
+
+### Migration
+
+None required. Public API, CLI flags, frontmatter shape, and
+`content_hash` outputs are all preserved bit-for-bit.
+
+## [1.0.6] — 2026-04-27
+
+Security hardening release. Closes 8 actionable findings from the
+2026-04-27 multi-tool audit (semgrep + trivy + gitleaks + bandit +
+pip-audit + manual review by the security-engineer agent + Opus 4.7).
+No public API changes; behavior unchanged for legitimate inputs.
+
+### Security
+
+- **F1 (HIGH)** SSRF: replaced `trafilatura.fetch_url` and the bare
+  `urllib.request.urlopen` paths with a new `any2md/_http.py` module
+  that does manual redirect walking with per-hop revalidation. Defends
+  against DNS-rebinding and redirect-based bypass of the IP-class
+  check. New `validate_url` rejects non-http(s) schemes and
+  private/loopback/link-local/reserved targets. 20 MB body cap, 3-hop
+  redirect cap, redirect-loop detection. Applied to `convert_url`,
+  `_http_last_modified`, and `arxiv_lookup`.
+- **F5 (MED)** `--meta` integrity: `content_hash`, `extracted_via`,
+  `source_file`, `lane`, `token_estimate`, `recommended_chunk_level`
+  are now **reserved** and silently filtered from `--meta` /
+  `--meta-file` / `.any2md.toml` overrides with a stderr `WARN` per
+  drop. Defense-in-depth filter also runs inside `frontmatter.compose`.
+- **F3 (MED)** TOML auto-discovery now stops at the first ancestor
+  containing `.git` / `pyproject.toml` / `setup.py` / `setup.cfg` /
+  `package.json` / `.any2md.toml.boundary`. New `--no-config` flag
+  fully disables discovery.
+- **F4 (MED)** DOCX zip-bomb: `_safe_zip_open` enforces a 1 MB cap on
+  the declared uncompressed size of `core.xml` and `app.xml` before
+  parsing. Oversized members raise `ValueError` → clean per-file FAIL.
+- **F-CVE (MED)** Dependency floors bumped: `lxml>=6.1.0,<7`
+  (CVE-2026-41066), `pillow>=12.2.0,<13` (CVE-2026-25990,
+  CVE-2026-40192), `urllib3>=2.6.3,<3` (CVE-2026-21441). All ranges
+  acquire upper bounds (compat envelope). `requirements.txt` becomes
+  a development/CI lockfile with exact pins.
+- **F6 (LOW)** Atomic writes: new `any2md.utils.atomic_write_text`
+  writes via `tempfile.mkstemp` + `os.replace`. Refuses to overwrite
+  a pre-existing symlink at the output path. Wired through
+  `pdf/docx/html/txt` converters; PDF image extraction adds a symlink
+  check before `pil_image.save`.
+- **F2 (LOW)** Control-char sanitizer strips C0/C1 controls (preserves
+  `\t`, `\n`) from captured Docling `msword_backend` warnings before
+  forwarding them to `collected_warnings()` and stderr. Defends
+  terminals from ANSI-escape spoofing.
+- **F11 (LOW)** PDF image-dir name now goes through `safe_dir_name`
+  (alphanumeric/`_`/`-` only). Closes a `Path.stem == ".."` corner
+  case that co-mingled images with markdown outputs.
+- **XXE defense-in-depth**: migrated `xml.etree.ElementTree` →
+  `defusedxml.ElementTree` in `docx.py` and `heuristics.py`. Stdlib
+  was already safe in Python 3.7+; this silences bandit B405/B314
+  permanently and protects against future stdlib regressions.
+
+### Added
+
+- New module `any2md/_http.py` (`validate_url`, `safe_fetch`).
+- New CLI flag `--no-config`.
+- New helpers `any2md.utils.atomic_write_text` and `safe_dir_name`.
+- New runtime dependency: `defusedxml>=0.7.1,<1`.
+
+### Tests
+
+- 7 new unit-test files (40 new tests): `test_http_safe.py`,
+  `test_meta_reserved_keys.py`, `test_config_walkup.py`,
+  `test_zipbomb_guard.py`, `test_atomic_write.py`,
+  `test_safe_dir_name.py`, `test_log_sanitizer.py`.
+- Existing 290 tests continue to pass.
+- Bandit audit went from **9 findings** (3 medium, 6 low) to **1 low**
+  (legitimate `try/except/pass` in arxiv `_warn`).
+- 5-DOCX regression run reproduces v1.0.5 word counts exactly
+  (978/6726/15566/7811/1477).
+
+### Audit summary
+
+| ID | Title | Severity |
+|----|-------|----------|
+| F1 | SSRF: DNS rebind + redirect bypass | High |
+| F5 | `--meta` clobbers reserved frontmatter | Medium |
+| F3 | TOML discovery walks above project root | Medium |
+| F4 | DOCX zip-bomb amplification | Medium |
+| F-CVE | Transitive deps with known CVEs | Medium |
+| F6 | Symlink-following on output writes | Low |
+| F2 | Control-char passthrough in Docling logs | Low |
+| F11 | PDF stem `..` corner case | Low |
+
 ## [1.0.5] — 2026-04-27
 
 Patch release. Recovers DOCX content that Docling's msword backend
@@ -235,7 +348,7 @@ Patch release. Adds an explicit backend-selection CLI flag.
 First stable release of any2md v1.0. Validated end-to-end against
 real-world documents (a 164-page PDF technical standard via Docling,
 a multi-page academic DOCX, and a Wikipedia article via trafilatura).
-The output contract — SSRM-compatible frontmatter, deterministic
+The output contract — SAGE-compatible frontmatter, deterministic
 `content_hash`, NFC + LF body normalization — is now stable. Downstream
 consumers can rely on this shape.
 
@@ -257,7 +370,7 @@ Phase 4: configuration, polish, and the v1.0 documentation set.
 - `--meta-file PATH` flag plus auto-discovery of `.any2md.toml` (walks up
   from cwd). The `[meta]` table supplies frontmatter overrides; the
   `[document_id]` table supplies `--auto-id` prefix and type code.
-- `--auto-id` flag — generates an SSRM-conformant `document_id` as
+- `--auto-id` flag — generates an SAGE-conformant `document_id` as
   `{PREFIX}-{YYYY}-{TYPE}-{SHA8}`. Defaults to `LOCAL`/`DOC`; override via
   the `[document_id]` table in `.any2md.toml`.
 - `--strict` flag — promotes pipeline validation warnings (heading
@@ -276,7 +389,7 @@ Phase 4: configuration, polish, and the v1.0 documentation set.
 - Comprehensive documentation set:
   - `README.md` rewritten with educational tone — every command preceded
     by why you'd run it and followed by what you'll see.
-  - `docs/output-format.md` — SSRM-compatible field reference and the
+  - `docs/output-format.md` — SAGE-compatible field reference and the
     `content_hash` recomputation recipe.
   - `docs/cli-reference.md` — flag-by-flag reference with use cases and a
     worked-example matrix.
@@ -379,14 +492,14 @@ any2md transparently falls back to pymupdf4llm (PDF) and mammoth (DOCX).
 
 First prerelease of any2md v1.0. Phase 1 of 5: foundation only — no Docling
 backend yet, no new CLI flags. Output frontmatter has been rewritten to be
-SSRM-compatible (Structured Security Reasoning Markdown) — this is a
+SAGE-compatible (Security Analysis and Guidance Exchange) — this is a
 breaking change for downstream consumers parsing v0.7 output. See
 `docs/superpowers/specs/2026-04-26-any2md-v1-design.md` for the full
 v1.0 design and `docs/superpowers/plans/2026-04-26-any2md-v1-phase1-foundation.md`
 for this phase's task plan.
 
 ### Added
-- New `any2md/frontmatter.py` module — SSRM-compatible YAML emitter with the
+- New `any2md/frontmatter.py` module — SAGE-compatible YAML emitter with the
   `SourceMeta` dataclass, deterministic `compute_content_hash`,
   `estimate_tokens`, `recommend_chunk_level`, `extract_abstract`, and
   `derive_title` helpers, plus the central `compose()` function (the only
@@ -412,7 +525,7 @@ for this phase's task plan.
   `reportlab`, `ruff`.
 
 ### Changed
-- **BREAKING:** Output frontmatter shape is now SSRM-compatible. New required
+- **BREAKING:** Output frontmatter shape is now SAGE-compatible. New required
   fields: `document_id` (empty string by default — opt-in via future
   `--auto-id`), `version`, `date`, `status: "draft"`, `document_type` (empty
   for non-security documents), `content_domain` (empty array),

{any2md-1.0.5 → any2md-1.0.7}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: any2md
-Version: 1.0.5
+Version: 1.0.7
 Summary: Convert PDF, DOCX, HTML, and TXT files — or web pages by URL — to clean, LLM-optimized Markdown with YAML frontmatter.
 Project-URL: Homepage, https://github.com/rocklambros/any2md
 Project-URL: Issues, https://github.com/rocklambros/any2md/issues
@@ -12,13 +12,16 @@ Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python :: 3
 Classifier: Topic :: Text Processing :: Markup :: Markdown
 Requires-Python: >=3.10
-Requires-Dist: beautifulsoup4>=4.12.0
-Requires-Dist: lxml>=5.0.0
-Requires-Dist: mammoth>=1.6.0
-Requires-Dist: markdownify>=0.13.0
-Requires-Dist: pymupdf4llm>=0.0.17
-Requires-Dist: pymupdf>=1.24.0
-Requires-Dist: trafilatura>=1.12.0
+Requires-Dist: beautifulsoup4<5,>=4.12.0
+Requires-Dist: defusedxml<1,>=0.7.1
+Requires-Dist: lxml<7,>=6.1.0
+Requires-Dist: mammoth<2,>=1.6.0
+Requires-Dist: markdownify<2,>=0.13.0
+Requires-Dist: pillow<13,>=12.2.0
+Requires-Dist: pymupdf4llm<1,>=0.0.17
+Requires-Dist: pymupdf<2,>=1.24.0
+Requires-Dist: trafilatura<3,>=1.12.0
+Requires-Dist: urllib3<3,>=2.6.3
 Provides-Extra: dev
 Requires-Dist: pytest-snapshot>=0.9.0; extra == 'dev'
 Requires-Dist: pytest>=8.0.0; extra == 'dev'
@@ -49,7 +52,7 @@ The full design rationale lives in [docs/superpowers/specs/2026-04-26-v1.0.3-emp
 
 ## What this is
 
-any2md ingests heterogeneous source documents and emits Markdown with a fixed YAML frontmatter contract. The frontmatter shape is SSRM-compatible (Structured Security Reasoning Markdown — a documented schema for LLM-consumable documents), so retrieval pipelines, embeddings jobs, and chunking utilities downstream can rely on field names, types, and a deterministic `content_hash` for cache invalidation. The body is NFC-normalized with LF line endings, and the heading hierarchy is guaranteed to start at H1 and not skip levels. The goal is one stable shape, regardless of whether a document started life as a scanned PDF, a Word doc, a Wikipedia page, or a plain text dump.
+any2md ingests heterogeneous source documents and emits Markdown with a fixed YAML frontmatter contract. The frontmatter shape is SAGE-compatible (Security Analysis and Guidance Exchange — a documented schema for LLM-consumable documents), so retrieval pipelines, embeddings jobs, and chunking utilities downstream can rely on field names, types, and a deterministic `content_hash` for cache invalidation. The body is NFC-normalized with LF line endings, and the heading hierarchy is guaranteed to start at H1 and not skip levels. The goal is one stable shape, regardless of whether a document started life as a scanned PDF, a Word doc, a Wikipedia page, or a plain text dump.
 
 ## Quick start
 
@@ -141,8 +144,8 @@ document_id: ""                                    # empty unless you pass --aut
 version: "1"                                       # "1" for source files without an embedded version
 date: "2026-03-15"                                 # docx core props "modified" → ISO-8601
 status: "draft"                                    # always "draft" for converted documents
-document_type: ""                                  # SSRM controlled vocab — empty for non-security docs
-content_domain: []                                 # SSRM controlled vocab — empty array when unknown
+document_type: ""                                  # SAGE controlled vocab — empty for non-security docs
+content_domain: []                                 # SAGE controlled vocab — empty array when unknown
 authors:                                           # extracted from docProps/core.xml dc:creator
   - "Rock Lambros"
 organization: ""                                   # docProps/app.xml Company; empty when absent
@@ -163,7 +166,7 @@ produced_by: "Microsoft® Word for Microsoft 365"   # software that produced the
 ---
 ```
 
-Fields under `# any2md extension field` are retained from v0.7 for traceability and observability — they're not part of the SSRM contract proper, but they live in the same frontmatter block. See [docs/output-format.md](docs/output-format.md) for the full field reference.
+Fields under `# any2md extension field` are retained from v0.7 for traceability and observability — they're not part of the SAGE contract proper, but they live in the same frontmatter block. See [docs/output-format.md](docs/output-format.md) for the full field reference.
 
 ### Before / after
 
@@ -359,7 +362,7 @@ You'll see the OK lines as if the conversion were fresh; previously-written outp
 
 ## The output format
 
-Every file converted by any2md has the same frontmatter shape, regardless of source format. The shape is **SSRM-compatible** — it matches the field names, types, and ordering of the Structured Security Reasoning Markdown specification, but values aren't required to match SSRM's controlled vocabularies (most converted documents aren't security research). Fields that require a controlled vocabulary (`document_type`, `content_domain`, `tlp`) are emitted empty unless you supply them via `--meta` or `.any2md.toml`.
+Every file converted by any2md has the same frontmatter shape, regardless of source format. The shape is **SAGE-compatible** — it matches the field names, types, and ordering of the Security Analysis and Guidance Exchange specification, but values aren't required to match SAGE's controlled vocabularies (most converted documents aren't security research). Fields that require a controlled vocabulary (`document_type`, `content_domain`, `tlp`) are emitted empty unless you supply them via `--meta` or `.any2md.toml`.
 
 ### Field auto-fill summary
 
@@ -404,7 +407,7 @@ You'll see the OK line as usual. The frontmatter of `Text/paper.md` will have `o
 
 ### `.any2md.toml`
 
-You'd use this when you have a stable set of frontmatter defaults that should apply to every conversion in a project. The file is auto-discovered by walking up from the current working directory, so you can drop one at the root of your corpus repo and forget about it. Worked example for a security-research org producing SSRM-conforming outputs:
+You'd use this when you have a stable set of frontmatter defaults that should apply to every conversion in a project. The file is auto-discovered by walking up from the current working directory, so you can drop one at the root of your corpus repo and forget about it. Worked example for a security-research org producing SAGE-conforming outputs:
 
 ```toml
 [meta]
@@ -558,7 +561,7 @@ The two-lane design exists because Docling output is layout-correct (running agg
 
 ## Migrating from v0.7
 
-v1.0 emits SSRM-compatible frontmatter — that's the one intended breaking change. v0.7's small frontmatter (`title`, `source_file`, `pages`, `type`, `word_count`) becomes a fuller block with `document_id`, `version`, `date`, `status`, `content_hash`, `token_estimate`, `authors`, and several other fields. The v0.7 fields are retained as any2md extension fields in the same block, so traceability is preserved, but downstream parsers that expected the old fixed shape will need to update. The body is now NFC-normalized with LF line endings, which means `content_hash` is reproducible. If you need bit-for-bit v0.7 output, pin `any2md==0.7.0`. The full field-by-field migration guide and behavior-change list lives in [docs/upgrading-from-0.7.md](docs/upgrading-from-0.7.md).
+v1.0 emits SAGE-compatible frontmatter — that's the one intended breaking change. v0.7's small frontmatter (`title`, `source_file`, `pages`, `type`, `word_count`) becomes a fuller block with `document_id`, `version`, `date`, `status`, `content_hash`, `token_estimate`, `authors`, and several other fields. The v0.7 fields are retained as any2md extension fields in the same block, so traceability is preserved, but downstream parsers that expected the old fixed shape will need to update. The body is now NFC-normalized with LF line endings, which means `content_hash` is reproducible. If you need bit-for-bit v0.7 output, pin `any2md==0.7.0`. The full field-by-field migration guide and behavior-change list lives in [docs/upgrading-from-0.7.md](docs/upgrading-from-0.7.md).
 
 ## Security
 

{any2md-1.0.5 → any2md-1.0.7}/README.md

@@ -19,7 +19,7 @@ The full design rationale lives in [docs/superpowers/specs/2026-04-26-v1.0.3-emp
 
 ## What this is
 
-any2md ingests heterogeneous source documents and emits Markdown with a fixed YAML frontmatter contract. The frontmatter shape is SSRM-compatible (Structured Security Reasoning Markdown — a documented schema for LLM-consumable documents), so retrieval pipelines, embeddings jobs, and chunking utilities downstream can rely on field names, types, and a deterministic `content_hash` for cache invalidation. The body is NFC-normalized with LF line endings, and the heading hierarchy is guaranteed to start at H1 and not skip levels. The goal is one stable shape, regardless of whether a document started life as a scanned PDF, a Word doc, a Wikipedia page, or a plain text dump.
+any2md ingests heterogeneous source documents and emits Markdown with a fixed YAML frontmatter contract. The frontmatter shape is SAGE-compatible (Security Analysis and Guidance Exchange — a documented schema for LLM-consumable documents), so retrieval pipelines, embeddings jobs, and chunking utilities downstream can rely on field names, types, and a deterministic `content_hash` for cache invalidation. The body is NFC-normalized with LF line endings, and the heading hierarchy is guaranteed to start at H1 and not skip levels. The goal is one stable shape, regardless of whether a document started life as a scanned PDF, a Word doc, a Wikipedia page, or a plain text dump.
 
 ## Quick start
 
@@ -111,8 +111,8 @@ document_id: ""                                    # empty unless you pass --aut
 version: "1"                                       # "1" for source files without an embedded version
 date: "2026-03-15"                                 # docx core props "modified" → ISO-8601
 status: "draft"                                    # always "draft" for converted documents
-document_type: ""                                  # SSRM controlled vocab — empty for non-security docs
-content_domain: []                                 # SSRM controlled vocab — empty array when unknown
+document_type: ""                                  # SAGE controlled vocab — empty for non-security docs
+content_domain: []                                 # SAGE controlled vocab — empty array when unknown
 authors:                                           # extracted from docProps/core.xml dc:creator
   - "Rock Lambros"
 organization: ""                                   # docProps/app.xml Company; empty when absent
@@ -133,7 +133,7 @@ produced_by: "Microsoft® Word for Microsoft 365"   # software that produced the
 ---
 ```
 
-Fields under `# any2md extension field` are retained from v0.7 for traceability and observability — they're not part of the SSRM contract proper, but they live in the same frontmatter block. See [docs/output-format.md](docs/output-format.md) for the full field reference.
+Fields under `# any2md extension field` are retained from v0.7 for traceability and observability — they're not part of the SAGE contract proper, but they live in the same frontmatter block. See [docs/output-format.md](docs/output-format.md) for the full field reference.
 
 ### Before / after
 
@@ -329,7 +329,7 @@ You'll see the OK lines as if the conversion were fresh; previously-written outp
 
 ## The output format
 
-Every file converted by any2md has the same frontmatter shape, regardless of source format. The shape is **SSRM-compatible** — it matches the field names, types, and ordering of the Structured Security Reasoning Markdown specification, but values aren't required to match SSRM's controlled vocabularies (most converted documents aren't security research). Fields that require a controlled vocabulary (`document_type`, `content_domain`, `tlp`) are emitted empty unless you supply them via `--meta` or `.any2md.toml`.
+Every file converted by any2md has the same frontmatter shape, regardless of source format. The shape is **SAGE-compatible** — it matches the field names, types, and ordering of the Security Analysis and Guidance Exchange specification, but values aren't required to match SAGE's controlled vocabularies (most converted documents aren't security research). Fields that require a controlled vocabulary (`document_type`, `content_domain`, `tlp`) are emitted empty unless you supply them via `--meta` or `.any2md.toml`.
 
 ### Field auto-fill summary
 
@@ -374,7 +374,7 @@ You'll see the OK line as usual. The frontmatter of `Text/paper.md` will have `o
 
 ### `.any2md.toml`
 
-You'd use this when you have a stable set of frontmatter defaults that should apply to every conversion in a project. The file is auto-discovered by walking up from the current working directory, so you can drop one at the root of your corpus repo and forget about it. Worked example for a security-research org producing SSRM-conforming outputs:
+You'd use this when you have a stable set of frontmatter defaults that should apply to every conversion in a project. The file is auto-discovered by walking up from the current working directory, so you can drop one at the root of your corpus repo and forget about it. Worked example for a security-research org producing SAGE-conforming outputs:
 
 ```toml
 [meta]
@@ -528,7 +528,7 @@ The two-lane design exists because Docling output is layout-correct (running agg
 
 ## Migrating from v0.7
 
-v1.0 emits SSRM-compatible frontmatter — that's the one intended breaking change. v0.7's small frontmatter (`title`, `source_file`, `pages`, `type`, `word_count`) becomes a fuller block with `document_id`, `version`, `date`, `status`, `content_hash`, `token_estimate`, `authors`, and several other fields. The v0.7 fields are retained as any2md extension fields in the same block, so traceability is preserved, but downstream parsers that expected the old fixed shape will need to update. The body is now NFC-normalized with LF line endings, which means `content_hash` is reproducible. If you need bit-for-bit v0.7 output, pin `any2md==0.7.0`. The full field-by-field migration guide and behavior-change list lives in [docs/upgrading-from-0.7.md](docs/upgrading-from-0.7.md).
+v1.0 emits SAGE-compatible frontmatter — that's the one intended breaking change. v0.7's small frontmatter (`title`, `source_file`, `pages`, `type`, `word_count`) becomes a fuller block with `document_id`, `version`, `date`, `status`, `content_hash`, `token_estimate`, `authors`, and several other fields. The v0.7 fields are retained as any2md extension fields in the same block, so traceability is preserved, but downstream parsers that expected the old fixed shape will need to update. The body is now NFC-normalized with LF line endings, which means `content_hash` is reproducible. If you need bit-for-bit v0.7 output, pin `any2md==0.7.0`. The full field-by-field migration guide and behavior-change list lives in [docs/upgrading-from-0.7.md](docs/upgrading-from-0.7.md).
 
 ## Security
 

{any2md-1.0.5 → any2md-1.0.7}/any2md/__init__.py

@@ -1,3 +1,3 @@
 """Convert PDF, DOCX, HTML, and TXT files to LLM-optimized Markdown."""
 
-__version__ = "1.0.5"
+__version__ = "1.0.7"

any2md-1.0.7/any2md/_http.py

@@ -0,0 +1,106 @@
+"""SSRF-safe HTTP fetching utilities for any2md.
+
+Centralizes URL scheme + IP validation and provides a fetcher that
+walks redirects manually and revalidates the host on each hop. This
+defends against:
+  - DNS rebinding (host -> public IP for validator, private IP for fetch)
+  - Redirect-based SSRF (public landing page -> 302 to 169.254.169.254)
+  - Non-http(s) schemes (file://, gopher://, etc.)
+  - Decompression / huge-response DoS (response-size cap)
+"""
+
+from __future__ import annotations
+
+import ipaddress
+import socket
+import urllib.error
+import urllib.parse
+import urllib.request
+
+_MAX_REDIRECT_HOPS = 3
+_FETCH_TIMEOUT = 15  # seconds
+_MAX_RESPONSE_BYTES = 20 * 1024 * 1024  # 20 MB cap on body read
+_USER_AGENT = "any2md/1.0.6"
+
+
+def validate_url(url: str) -> str | None:
+    """Validate URL scheme + host. Returns an error message or None.
+
+    One-shot DNS lookup; rebind protection requires re-checking on
+    every redirect hop (handled by ``safe_fetch``).
+    """
+    parsed = urllib.parse.urlparse(url)
+    if parsed.scheme not in ("http", "https"):
+        return f"unsupported scheme: {parsed.scheme!r}"
+    if not parsed.hostname:
+        return f"no hostname in URL: {url}"
+    try:
+        infos = socket.getaddrinfo(parsed.hostname, None, type=socket.SOCK_STREAM)
+    except socket.gaierror:
+        return f"cannot resolve host: {parsed.hostname}"
+    for *_, sockaddr in infos:
+        try:
+            addr = ipaddress.ip_address(sockaddr[0])
+        except ValueError:
+            continue
+        if (
+            addr.is_private
+            or addr.is_reserved
+            or addr.is_loopback
+            or addr.is_link_local
+        ):
+            return f"URL resolves to disallowed address: {addr}"
+    return None
+
+
+class _NoFollowRedirect(urllib.request.HTTPRedirectHandler):
+    """Disable urllib's automatic redirect following."""
+
+    def redirect_request(self, *args, **kwargs):  # noqa: ARG002
+        return None
+
+
+def safe_fetch(
+    url: str,
+    *,
+    method: str = "GET",
+    max_hops: int = _MAX_REDIRECT_HOPS,
+) -> tuple[bytes | None, dict | None, str | None]:
+    """Fetch ``url`` with manual redirect walking + per-hop revalidation.
+
+    Returns ``(body, headers, None)`` on 2xx, or ``(None, None, error)``
+    on rejection / failure. ``headers`` is the response headers as a
+    plain ``dict``.
+    """
+    visited: list[str] = []
+    current = url
+    opener = urllib.request.build_opener(_NoFollowRedirect())
+    for hop in range(max_hops + 1):
+        if current in visited:
+            return None, None, "redirect loop"
+        visited.append(current)
+        err = validate_url(current)
+        if err:
+            return None, None, err
+        req = urllib.request.Request(
+            current, method=method, headers={"User-Agent": _USER_AGENT}
+        )
+        try:
+            resp = opener.open(req, timeout=_FETCH_TIMEOUT)
+        except urllib.error.HTTPError as e:
+            if e.code in (301, 302, 303, 307, 308):
+                if hop >= max_hops:
+                    return None, None, f"too many redirects (>{max_hops})"
+                location = e.headers.get("Location") if e.headers else None
+                if not location:
+                    return None, None, f"HTTP {e.code} without Location"
+                current = urllib.parse.urljoin(current, location)
+                continue
+            return None, None, f"HTTP {e.code}"
+        except (urllib.error.URLError, OSError, TimeoutError) as e:
+            return None, None, f"fetch error: {e}"
+        body = resp.read(_MAX_RESPONSE_BYTES + 1)
+        if len(body) > _MAX_RESPONSE_BYTES:
+            return None, None, f"response exceeds {_MAX_RESPONSE_BYTES} bytes"
+        return body, dict(resp.headers), None
+    return None, None, f"too many redirects (>{max_hops})"

{any2md-1.0.5 → any2md-1.0.7}/any2md/cli.py

@@ -20,6 +20,7 @@ from any2md.converters import (
     set_output_mode,
 )
 from any2md.converters.html import convert_url
+from any2md.frontmatter import filter_reserved_overrides
 from any2md.pipeline import PipelineOptions
 from any2md.utils import sanitize_filename, url_to_filename
 
@@ -169,6 +170,11 @@ def main():
         action="store_true",
         help="Promote pipeline validation warnings to errors (exit 3).",
     )
+    parser.add_argument(
+        "--no-config",
+        action="store_true",
+        help="Skip .any2md.toml auto-discovery (overrides default walk-up).",
+    )
     parser.add_argument(
         "--docx-fallback-on-warn",
         action=argparse.BooleanOptionalAction,
@@ -194,7 +200,7 @@ def main():
     parser.add_argument(
         "--auto-id",
         action="store_true",
-        help="Generate an SSRM-conformant document_id of the form "
+        help="Generate an SAGE-conformant document_id of the form "
         "{PREFIX}-{YYYY}-{TYPE}-{SHA8}. Defaults: PREFIX=LOCAL, TYPE=DOC. "
         "Override via .any2md.toml [document_id] (publisher_prefix, type_code).",
     )
@@ -222,7 +228,7 @@ def main():
     overrides: dict[str, Any] = {}
     auto_id_prefix, auto_id_type_code = "LOCAL", "DOC"
 
-    discovered = discover_config()
+    discovered = None if args.no_config else discover_config()
     if discovered is not None:
         cfg = load_toml(discovered)
         overrides = _deep_merge_overrides(overrides, extract_meta_overrides(cfg))
@@ -249,6 +255,9 @@ def main():
         print(f"Error: {e}", file=sys.stderr)
         sys.exit(1)
     overrides = _deep_merge_overrides(overrides, cli_meta)
+    overrides = filter_reserved_overrides(
+        overrides, source_label="--meta/--meta-file/.any2md.toml"
+    )
 
     if (
         args.high_fidelity

{any2md-1.0.5 → any2md-1.0.7}/any2md/config.py

@@ -42,17 +42,31 @@ def load_toml(path: Path) -> dict[str, Any]:
         return {}
 
 
+_PROJECT_BOUNDARY_MARKERS = (
+    ".git",
+    "pyproject.toml",
+    "setup.py",
+    "setup.cfg",
+    "package.json",
+    ".any2md.toml.boundary",
+)
+
+
 def discover_config(start: Path | None = None) -> Path | None:
     """Walk up from ``start`` (default: cwd) looking for ``.any2md.toml``.
 
-    Returns the first match found or ``None`` if no config exists in
-    any ancestor directory.
+    Stops at the first ancestor containing a project boundary marker
+    (``.git``, ``pyproject.toml``, ``setup.py``, ``setup.cfg``,
+    ``package.json``, or ``.any2md.toml.boundary``) so config in
+    unrelated parent directories cannot leak into a project's run.
     """
     cur = (start or Path.cwd()).resolve()
     while True:
         candidate = cur / ".any2md.toml"
         if candidate.is_file():
             return candidate
+        if any((cur / m).exists() for m in _PROJECT_BOUNDARY_MARKERS):
+            return None
         if cur.parent == cur:
             return None
         cur = cur.parent

{any2md-1.0.5 → any2md-1.0.7}/any2md/converters/docx.py

@@ -10,9 +10,12 @@ produces the markdown body.
 from __future__ import annotations
 
 import logging
+import re
 import sys
-import xml.etree.ElementTree as ET
 import zipfile
+
+from defusedxml.ElementTree import ParseError as _XmlParseError
+from defusedxml.ElementTree import parse as _xml_parse
 from datetime import date
 from pathlib import Path
 
@@ -25,11 +28,44 @@ from any2md.converters import add_warnings, is_quiet
 from any2md.frontmatter import SourceMeta, compose
 from any2md.heuristics import filter_organization
 from any2md.pipeline import PipelineOptions
-from any2md.utils import sanitize_filename
+from any2md.utils import atomic_write_text, sanitize_filename
 
 
 _DOCLING_MSWORD_LOGGER = "docling.backend.msword_backend"
 
+_LOG_CONTROL_CHARS_RE = re.compile(r"[\x00-\x08\x0b\x0c\x0e-\x1f\x7f-\x9f]")
+
+
+def _sanitize_log_text(s: str) -> str:
+    """Strip ASCII C0/C1 control chars from a log message before re-emission.
+
+    Defends terminals from ANSI-escape spoofing planted in malicious DOCX
+    content that surfaces as a Docling msword_backend warning. ``\\t``
+    and ``\\n`` are preserved (not in the strip class).
+    """
+    return _LOG_CONTROL_CHARS_RE.sub("", s)
+
+
+_MAX_DOCX_METADATA_SIZE = 1 * 1024 * 1024  # 1 MB; core.xml/app.xml are typically <10 KB
+
+
+def _safe_zip_open(
+    z: zipfile.ZipFile, name: str, max_size: int = _MAX_DOCX_METADATA_SIZE
+):
+    """Open a fixed-name DOCX member after a declared-size sanity check.
+
+    Defends against zip-bomb amplification: a malicious DOCX can declare
+    an uncompressed_size of 10 GB for ``core.xml`` and bomb the XML
+    parser. Raises ``ValueError`` (caught by ``convert_docx``'s existing
+    handler -> file fails cleanly).
+    """
+    info = z.getinfo(name)
+    if info.file_size > max_size:
+        raise ValueError(
+            f"{name} declared size {info.file_size} exceeds {max_size}-byte limit"
+        )
+    return z.open(name)
+
 
 class _DoclingMswordWarningCapture:
     """Buffer WARNING+ records from Docling's DOCX backend logger.
@@ -89,8 +125,8 @@ def _read_docx_metadata(docx_path: Path) -> dict[str, object]:
     try:
         with zipfile.ZipFile(docx_path) as z:
             try:
-                with z.open("docProps/core.xml") as f:
-                    root = ET.parse(f).getroot()
+                with _safe_zip_open(z, "docProps/core.xml") as f:
+                    root = _xml_parse(f).getroot()
                 title = root.findtext("dc:title", namespaces=_NS_CORE)
                 if title:
                     out["title_hint"] = title.strip()
@@ -105,8 +141,8 @@ def _read_docx_metadata(docx_path: Path) -> dict[str, object]:
             except KeyError:
                 pass
             try:
-                with z.open("docProps/app.xml") as f:
-                    root = ET.parse(f).getroot()
+                with _safe_zip_open(z, "docProps/app.xml") as f:
+                    root = _xml_parse(f).getroot()
                 company = root.findtext("ext:Company", namespaces=_NS_APP)
                 application = root.findtext("ext:Application", namespaces=_NS_APP)
                 # v1.0.2: Company takes priority for `organization` (real
@@ -126,7 +162,7 @@ def _read_docx_metadata(docx_path: Path) -> dict[str, object]:
                     out["produced_by"] = app_result.produced_by
             except KeyError:
                 pass
-    except (zipfile.BadZipFile, ET.ParseError):
+    except (zipfile.BadZipFile, _XmlParseError):
         pass
     return out
 
@@ -233,7 +269,12 @@ def convert_docx(
             md_text, _ = _extract_via_mammoth(docx_path, options)
             extracted_via = "docling→mammoth (warning fallback)"
             lane = "text"
-            add_warnings([f"docling.msword_backend: {m}" for m in docling_warnings])
+            add_warnings(
+                [
+                    f"docling.msword_backend: {_sanitize_log_text(m)}"
+                    for m in docling_warnings
+                ]
+            )
 
         md_text, warnings = pipeline.run(md_text, lane, options)
         add_warnings(warnings)
@@ -260,7 +301,7 @@ def convert_docx(
         full = compose(md_text, meta, options, overrides=options.frontmatter_overrides)
 
         output_dir.mkdir(parents=True, exist_ok=True)
-        out_path.write_text(full, encoding="utf-8", newline="\n")
+        atomic_write_text(out_path, full)
         wc = meta.word_count or 0
         suffix = f", {len(warnings)} warning(s)" if warnings else ""
         if not is_quiet():