any2md 1.0.5__tar.gz → 1.1.1__tar.gz

{any2md-1.0.5 → any2md-1.1.1}/.gitignore

@@ -43,4 +43,3 @@ logo/
 # ZERG / dev tooling
 .gsd/
 .zerg/
-.devcontainer/

{any2md-1.0.5 → any2md-1.1.1}/CHANGELOG.md

@@ -4,6 +4,255 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
+## [1.1.1] — 2026-05-DD
+
+Security patch release. Closes 10 findings from a comprehensive scan
+(bandit, snyk-code, gitleaks, pip-audit, trivy, manual review) plus
+a 5-round adversarial review of the implementation plan.
+
+### Security
+
+- **SSRF intra-hop DNS-rebind**: `safe_fetch` now resolves the
+  hostname once, validates the IP against an allowlist, and passes
+  the validated IP through to `socket.create_connection` via a
+  custom `HTTPSConnection`. Defeats attacks that return a public IP
+  to a validator and a private IP (e.g., `169.254.169.254`) to the
+  connector. `ProxyHandler({})` explicitly suppresses
+  `HTTP_PROXY`/`HTTPS_PROXY` env vars that would otherwise route
+  traffic past the validator.
+- **IP allowlist**: switched from a four-flag denylist
+  (`is_private | is_reserved | is_loopback | is_link_local`) to an
+  allowlist (`is_global and not is_multicast and not is_unspecified`).
+  **Breaking** for users on multicast / CGNAT / IETF-reserved
+  networks fetching same-network resources; pre-fetch with `curl`
+  and pass `--from-file` as a workaround.
+- **Frontmatter override-key validation**: keys must now match
+  `^[A-Za-z_][A-Za-z0-9_-]*$`. Defends against newline-injection in
+  TOML basic-string keys that produced split YAML lines.
+  **Breaking** for TOML configs with quoted keys containing
+  whitespace or punctuation.
+- **URL credentials and sensitive query parameters** in `userinfo`
+  and matching `(?i)(token|api[_-]?key|access[_-]?token|bearer|signature|sig|password|secret|aws[_-]?session[_-]?token)$`
+  are scrubbed before any logging, persistence, or filename
+  derivation. IPv6 host brackets preserved across the scrub.
+- **Docling image saves** use `O_EXCL|O_NOFOLLOW` to refuse symlink
+  TOCTOU.
+- **`atomic_write_text`** tightens the symlink-check window via
+  `os.lstat` immediately before `os.replace`. Parent-dir hardening
+  deferred to v1.2 per the operator-trusted threat model.
+- **Stderr/stdout sanitizer** extended with Unicode bidi
+  (U+202A–E, U+2066–9), zero-width chars (U+200B–F, U+FEFF), and
+  the BOM. New `any2md._logging` module; `_sanitize_log_text`
+  re-exported from `any2md.converters.docx` for one release.
+- **`publish.yml` workflow**: top-level `permissions: contents: read`;
+  both publish jobs flip to `attestations: true` for PEP 740 sigstore
+  provenance.
+
+### Fixed
+
+- `wikipedia.org` heuristic host check no longer matches
+  `evilwikipedia.org`. Pure correctness fix; impact was incorrect
+  text formatting only, not a security boundary.
+- `_USER_AGENT` no longer hardcoded to `any2md/1.0.6`; derives from
+  `__version__`.
+
+### Changed
+
+- `requirements.txt` is now a generated artifact; canonical source
+  is `.devcontainer/requirements.lock` with hashes
+  (`--require-hashes` enforced in `.devcontainer/devcontainer.json`).
+  See `CONTRIBUTING.md` for the dep-update workflow.
+- `load_toml()` now warns to stderr on parse error rather than
+  silently returning `{}`.
+- `.devcontainer/` is now tracked in git (was previously ignored)
+  so the security-critical Codespace bootstrap config is
+  version-controlled.
+
+## [1.1.0] — 2026-05-04
+
+Performance and library-mode hygiene release.
+
+### Added
+
+- `any2md.release_models()` — imperative escape hatch to free
+  cached Docling `DocumentConverter` instances. Stability:
+  experimental for v1.1.0.
+- `any2md.docling_session()` — contextmanager (preferred shape
+  for library use). Stability: experimental for v1.1.0.
+- `ANY2MD_DOCLING_CACHE=0` env-var to disable the persistent
+  converter cache (per-call construction, equivalent to v1.0.x
+  behavior).
+- `__all__` declared on `any2md/__init__.py` (formalizes the
+  public surface).
+- New benchmark `scripts/bench_docling_cache.py` for maintainer
+  pre-release verification.
+
+### Changed
+
+- `DocumentConverter` is now constructed once per process and
+  reused across all files in a batch (was: once per file). New
+  one-time stderr line `Loading Docling models (one-time)...`
+  printed on first PDF/DOCX in non-quiet TTY runs.
+- `.github/workflows/ci.yml` now runs `pytest tests/unit` on each
+  push and PR (was: lint + import smoke + pip-audit only).
+
+### Performance
+
+- Measured: CPU first-call ~2.47s (model load), warm calls ~0.87s
+  (vs prior ~2.47s per file in v1.0.x). Mac MPS first-call ~2.77s,
+  warm ~0.14s. RSS floor +535 MB sustained for the process lifetime
+  (see `docs/troubleshooting.md` for `release_models()` mitigation).
+
+### Migration
+
+- None required for CLI users. The `Loading Docling models...`
+  stderr line is the only user-visible default-output change.
+- Library/embedder users holding any2md across long-running
+  processes should call `any2md.release_models()` or use
+  `with any2md.docling_session():` to free model state between
+  workloads.
+- Test/instrumentation code that monkey-patches
+  `docling.document_converter.DocumentConverter` AFTER any2md's
+  first call will see no effect on subsequent calls (cache returns
+  the pre-patch instance). Workaround: call `release_models()`
+  after applying the patch, or set `ANY2MD_DOCLING_CACHE=0` in the
+  test environment.
+
+### Stability notice
+
+The new public APIs (`release_models`, `docling_session`) are
+marked experimental for v1.1.0; their shape may change in
+subsequent v1.x releases without a major bump until promoted to
+stable in a later release.
+
+### Known limitations
+
+- HF Hub outages during cold-start now cause N retries instead of
+  one silent fallback (the v1.0.x failure-cache behavior was
+  rejected on quality grounds). Bounded retry-with-backoff
+  deferred to a future hardening PR.
+- Forking a process (`multiprocessing` with `fork` start method)
+  after the first `convert()` call inherits torch/CUDA state
+  unsafely regardless of cache. Use
+  `multiprocessing.set_start_method("spawn")` or call
+  `release_models()` before fork.
+- `os.register_at_fork` is POSIX-only; on Windows the fork-reset
+  path is dead code (no fork on Windows). Cache works normally
+  on Windows.
+
+## [1.0.7] — 2026-04-29
+
+Branding/rebrand release. The frontmatter contract that any2md emits is
+now described as "SAGE-compatible" (Security Analysis and Guidance
+Exchange) — the upstream specification was renamed from SSRM (Structured
+Security Reasoning Markdown) to SAGE. No functional or schema changes:
+field names, ordering, controlled vocabularies, and `content_hash`
+derivation are byte-identical to v1.0.6. Existing v1.0.6 outputs remain
+valid SAGE-compatible documents without modification.
+
+### Changed
+
+- README, CHANGELOG, and all of `docs/` updated to reference SAGE /
+  Security Analysis and Guidance Exchange.
+- `--auto-id` CLI help text now reads "SAGE-conformant document_id"
+  (previously "SSRM-conformant"). Behavior unchanged.
+- Module/function docstrings in `any2md/frontmatter.py`,
+  `any2md/converters/txt.py`, and `any2md/pipeline/cleanup.py` updated
+  to reference SAGE §3 / §5.1 (same section numbers as the upstream
+  SSRM v1.0-RC1 spec).
+- Two test functions renamed for naming consistency
+  (`test_txt_end_to_end_writes_sage_compat_output`,
+  `test_compose_emits_required_sage_fields`). Pytest auto-discovery is
+  unaffected; assertions unchanged.
+
+### Migration
+
+None required. Public API, CLI flags, frontmatter shape, and
+`content_hash` outputs are all preserved bit-for-bit.
+
+## [1.0.6] — 2026-04-27
+
+Security hardening release. Closes 8 actionable findings from the
+2026-04-27 multi-tool audit (semgrep + trivy + gitleaks + bandit +
+pip-audit + manual review by the security-engineer agent + Opus 4.7).
+No public API changes; behavior unchanged for legitimate inputs.
+
+### Security
+
+- **F1 (HIGH)** SSRF: replaced `trafilatura.fetch_url` and the bare
+  `urllib.request.urlopen` paths with a new `any2md/_http.py` module
+  that does manual redirect walking with per-hop revalidation. Defends
+  against DNS-rebinding and redirect-based bypass of the IP-class
+  check. New `validate_url` rejects non-http(s) schemes and
+  private/loopback/link-local/reserved targets. 20 MB body cap, 3-hop
+  redirect cap, redirect-loop detection. Applied to `convert_url`,
+  `_http_last_modified`, and `arxiv_lookup`.
+- **F5 (MED)** `--meta` integrity: `content_hash`, `extracted_via`,
+  `source_file`, `lane`, `token_estimate`, `recommended_chunk_level`
+  are now **reserved** and silently filtered from `--meta` /
+  `--meta-file` / `.any2md.toml` overrides with a stderr `WARN` per
+  drop. Defense-in-depth filter also runs inside `frontmatter.compose`.
+- **F3 (MED)** TOML auto-discovery now stops at the first ancestor
+  containing `.git` / `pyproject.toml` / `setup.py` / `setup.cfg` /
+  `package.json` / `.any2md.toml.boundary`. New `--no-config` flag
+  fully disables discovery.
+- **F4 (MED)** DOCX zip-bomb: `_safe_zip_open` enforces a 1 MB cap on
+  the declared uncompressed size of `core.xml` and `app.xml` before
+  parsing. Oversized members raise `ValueError` → clean per-file FAIL.
+- **F-CVE (MED)** Dependency floors bumped: `lxml>=6.1.0,<7`
+  (CVE-2026-41066), `pillow>=12.2.0,<13` (CVE-2026-25990,
+  CVE-2026-40192), `urllib3>=2.6.3,<3` (CVE-2026-21441). All ranges
+  acquire upper bounds (compat envelope). `requirements.txt` becomes
+  a development/CI lockfile with exact pins.
+- **F6 (LOW)** Atomic writes: new `any2md.utils.atomic_write_text`
+  writes via `tempfile.mkstemp` + `os.replace`. Refuses to overwrite
+  a pre-existing symlink at the output path. Wired through
+  `pdf/docx/html/txt` converters; PDF image extraction adds a symlink
+  check before `pil_image.save`.
+- **F2 (LOW)** Control-char sanitizer strips C0/C1 controls (preserves
+  `\t`, `\n`) from captured Docling `msword_backend` warnings before
+  forwarding them to `collected_warnings()` and stderr. Defends
+  terminals from ANSI-escape spoofing.
+- **F11 (LOW)** PDF image-dir name now goes through `safe_dir_name`
+  (alphanumeric/`_`/`-` only). Closes a `Path.stem == ".."` corner
+  case that co-mingled images with markdown outputs.
+- **XXE defense-in-depth**: migrated `xml.etree.ElementTree` →
+  `defusedxml.ElementTree` in `docx.py` and `heuristics.py`. Stdlib
+  was already safe in Python 3.7+; this silences bandit B405/B314
+  permanently and protects against future stdlib regressions.
+
+### Added
+
+- New module `any2md/_http.py` (`validate_url`, `safe_fetch`).
+- New CLI flag `--no-config`.
+- New helpers `any2md.utils.atomic_write_text` and `safe_dir_name`.
+- New runtime dependency: `defusedxml>=0.7.1,<1`.
+
+### Tests
+
+- 7 new unit-test files (40 new tests): `test_http_safe.py`,
+  `test_meta_reserved_keys.py`, `test_config_walkup.py`,
+  `test_zipbomb_guard.py`, `test_atomic_write.py`,
+  `test_safe_dir_name.py`, `test_log_sanitizer.py`.
+- Existing 290 tests continue to pass.
+- Bandit audit went from **9 findings** (3 medium, 6 low) to **1 low**
+  (legitimate `try/except/pass` in arxiv `_warn`).
+- 5-DOCX regression run reproduces v1.0.5 word counts exactly
+  (978/6726/15566/7811/1477).
+
+### Audit summary
+
+| ID | Title | Severity |
+|----|-------|----------|
+| F1 | SSRF: DNS rebind + redirect bypass | High |
+| F5 | `--meta` clobbers reserved frontmatter | Medium |
+| F3 | TOML discovery walks above project root | Medium |
+| F4 | DOCX zip-bomb amplification | Medium |
+| F-CVE | Transitive deps with known CVEs | Medium |
+| F6 | Symlink-following on output writes | Low |
+| F2 | Control-char passthrough in Docling logs | Low |
+| F11 | PDF stem `..` corner case | Low |
+
 ## [1.0.5] — 2026-04-27
 
 Patch release. Recovers DOCX content that Docling's msword backend
@@ -235,7 +484,7 @@ Patch release. Adds an explicit backend-selection CLI flag.
 First stable release of any2md v1.0. Validated end-to-end against
 real-world documents (a 164-page PDF technical standard via Docling,
 a multi-page academic DOCX, and a Wikipedia article via trafilatura).
-The output contract — SSRM-compatible frontmatter, deterministic
+The output contract — SAGE-compatible frontmatter, deterministic
 `content_hash`, NFC + LF body normalization — is now stable. Downstream
 consumers can rely on this shape.
 
@@ -257,7 +506,7 @@ Phase 4: configuration, polish, and the v1.0 documentation set.
 - `--meta-file PATH` flag plus auto-discovery of `.any2md.toml` (walks up
   from cwd). The `[meta]` table supplies frontmatter overrides; the
   `[document_id]` table supplies `--auto-id` prefix and type code.
-- `--auto-id` flag — generates an SSRM-conformant `document_id` as
+- `--auto-id` flag — generates an SAGE-conformant `document_id` as
   `{PREFIX}-{YYYY}-{TYPE}-{SHA8}`. Defaults to `LOCAL`/`DOC`; override via
   the `[document_id]` table in `.any2md.toml`.
 - `--strict` flag — promotes pipeline validation warnings (heading
@@ -276,7 +525,7 @@ Phase 4: configuration, polish, and the v1.0 documentation set.
 - Comprehensive documentation set:
   - `README.md` rewritten with educational tone — every command preceded
     by why you'd run it and followed by what you'll see.
-  - `docs/output-format.md` — SSRM-compatible field reference and the
+  - `docs/output-format.md` — SAGE-compatible field reference and the
     `content_hash` recomputation recipe.
   - `docs/cli-reference.md` — flag-by-flag reference with use cases and a
     worked-example matrix.
@@ -379,14 +628,14 @@ any2md transparently falls back to pymupdf4llm (PDF) and mammoth (DOCX).
 
 First prerelease of any2md v1.0. Phase 1 of 5: foundation only — no Docling
 backend yet, no new CLI flags. Output frontmatter has been rewritten to be
-SSRM-compatible (Structured Security Reasoning Markdown) — this is a
+SAGE-compatible (Security Analysis and Guidance Exchange) — this is a
 breaking change for downstream consumers parsing v0.7 output. See
 `docs/superpowers/specs/2026-04-26-any2md-v1-design.md` for the full
 v1.0 design and `docs/superpowers/plans/2026-04-26-any2md-v1-phase1-foundation.md`
 for this phase's task plan.
 
 ### Added
-- New `any2md/frontmatter.py` module — SSRM-compatible YAML emitter with the
+- New `any2md/frontmatter.py` module — SAGE-compatible YAML emitter with the
   `SourceMeta` dataclass, deterministic `compute_content_hash`,
   `estimate_tokens`, `recommend_chunk_level`, `extract_abstract`, and
   `derive_title` helpers, plus the central `compose()` function (the only
@@ -412,7 +661,7 @@ for this phase's task plan.
   `reportlab`, `ruff`.
 
 ### Changed
-- **BREAKING:** Output frontmatter shape is now SSRM-compatible. New required
+- **BREAKING:** Output frontmatter shape is now SAGE-compatible. New required
   fields: `document_id` (empty string by default — opt-in via future
   `--auto-id`), `version`, `date`, `status: "draft"`, `document_type` (empty
   for non-security documents), `content_domain` (empty array),

{any2md-1.0.5 → any2md-1.1.1}/CONTRIBUTING.md

@@ -131,6 +131,24 @@ The flow:
 Don't tag and release directly without the PR — CI gates exist for a
 reason.
 
+## Updating dependencies
+
+Dependencies are declared in `pyproject.toml` and locked into both:
+
+- `.devcontainer/requirements.lock` — hash-locked, used by Codespace `postCreateCommand` (`pip install --require-hashes`).
+- `requirements.txt` — non-hash form, kept for backwards compatibility with `pip install -r requirements.txt` consumers.
+
+To add or update a dep:
+
+1. Edit `pyproject.toml` `[project] dependencies` (or the relevant `[project.optional-dependencies]` group).
+2. Regenerate both lockfiles in a venv with `pip-tools` installed:
+
+       pip-compile pyproject.toml --generate-hashes --resolver=backtracking \
+         -o .devcontainer/requirements.lock
+       pip-compile pyproject.toml --resolver=backtracking -o requirements.txt
+
+3. Commit all three files together.
+
 ## Pull request process
 
 - Branch off `main` (or off a phase branch like `v1.0` if one is open for

{any2md-1.0.5 → any2md-1.1.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: any2md
-Version: 1.0.5
+Version: 1.1.1
 Summary: Convert PDF, DOCX, HTML, and TXT files — or web pages by URL — to clean, LLM-optimized Markdown with YAML frontmatter.
 Project-URL: Homepage, https://github.com/rocklambros/any2md
 Project-URL: Issues, https://github.com/rocklambros/any2md/issues
@@ -12,16 +12,21 @@ Classifier: Operating System :: OS Independent
 Classifier: Programming Language :: Python :: 3
 Classifier: Topic :: Text Processing :: Markup :: Markdown
 Requires-Python: >=3.10
-Requires-Dist: beautifulsoup4>=4.12.0
-Requires-Dist: lxml>=5.0.0
-Requires-Dist: mammoth>=1.6.0
-Requires-Dist: markdownify>=0.13.0
-Requires-Dist: pymupdf4llm>=0.0.17
-Requires-Dist: pymupdf>=1.24.0
-Requires-Dist: trafilatura>=1.12.0
+Requires-Dist: beautifulsoup4<5,>=4.12.0
+Requires-Dist: defusedxml<1,>=0.7.1
+Requires-Dist: lxml<7,>=6.1.0
+Requires-Dist: mammoth<2,>=1.6.0
+Requires-Dist: markdownify<2,>=0.13.0
+Requires-Dist: pillow<13,>=12.2.0
+Requires-Dist: pymupdf4llm<1,>=0.0.17
+Requires-Dist: pymupdf<2,>=1.24.0
+Requires-Dist: tomli>=2.0.0; python_version < '3.11'
+Requires-Dist: trafilatura<3,>=1.12.0
+Requires-Dist: urllib3<3,>=2.6.3
 Provides-Extra: dev
 Requires-Dist: pytest-snapshot>=0.9.0; extra == 'dev'
 Requires-Dist: pytest>=8.0.0; extra == 'dev'
+Requires-Dist: pyyaml>=6.0; extra == 'dev'
 Requires-Dist: reportlab>=4.0.0; extra == 'dev'
 Requires-Dist: ruff>=0.6.0; extra == 'dev'
 Provides-Extra: high-fidelity
@@ -49,7 +54,7 @@ The full design rationale lives in [docs/superpowers/specs/2026-04-26-v1.0.3-emp
 
 ## What this is
 
-any2md ingests heterogeneous source documents and emits Markdown with a fixed YAML frontmatter contract. The frontmatter shape is SSRM-compatible (Structured Security Reasoning Markdown — a documented schema for LLM-consumable documents), so retrieval pipelines, embeddings jobs, and chunking utilities downstream can rely on field names, types, and a deterministic `content_hash` for cache invalidation. The body is NFC-normalized with LF line endings, and the heading hierarchy is guaranteed to start at H1 and not skip levels. The goal is one stable shape, regardless of whether a document started life as a scanned PDF, a Word doc, a Wikipedia page, or a plain text dump.
+any2md ingests heterogeneous source documents and emits Markdown with a fixed YAML frontmatter contract. The frontmatter shape is SAGE-compatible (Security Analysis and Guidance Exchange — a documented schema for LLM-consumable documents), so retrieval pipelines, embeddings jobs, and chunking utilities downstream can rely on field names, types, and a deterministic `content_hash` for cache invalidation. The body is NFC-normalized with LF line endings, and the heading hierarchy is guaranteed to start at H1 and not skip levels. The goal is one stable shape, regardless of whether a document started life as a scanned PDF, a Word doc, a Wikipedia page, or a plain text dump.
 
 ## Quick start
 
@@ -141,8 +146,8 @@ document_id: ""                                    # empty unless you pass --aut
 version: "1"                                       # "1" for source files without an embedded version
 date: "2026-03-15"                                 # docx core props "modified" → ISO-8601
 status: "draft"                                    # always "draft" for converted documents
-document_type: ""                                  # SSRM controlled vocab — empty for non-security docs
-content_domain: []                                 # SSRM controlled vocab — empty array when unknown
+document_type: ""                                  # SAGE controlled vocab — empty for non-security docs
+content_domain: []                                 # SAGE controlled vocab — empty array when unknown
 authors:                                           # extracted from docProps/core.xml dc:creator
   - "Rock Lambros"
 organization: ""                                   # docProps/app.xml Company; empty when absent
@@ -163,7 +168,7 @@ produced_by: "Microsoft® Word for Microsoft 365"   # software that produced the
 ---
 ```
 
-Fields under `# any2md extension field` are retained from v0.7 for traceability and observability — they're not part of the SSRM contract proper, but they live in the same frontmatter block. See [docs/output-format.md](docs/output-format.md) for the full field reference.
+Fields under `# any2md extension field` are retained from v0.7 for traceability and observability — they're not part of the SAGE contract proper, but they live in the same frontmatter block. See [docs/output-format.md](docs/output-format.md) for the full field reference.
 
 ### Before / after
 
@@ -357,9 +362,18 @@ any2md -f -r ./corpus/
 
 You'll see the OK lines as if the conversion were fresh; previously-written outputs are overwritten.
 
+> **v1.1.0+:** when processing PDFs or DOCX files via the Docling
+> backend, the converter (and its loaded model weights) is constructed
+> once on the first PDF/DOCX in an invocation and reused for all
+> subsequent files; subsequent files extract in ~0.5–1s rather than
+> re-paying the ~2.5s cold-load tax. Library users embedding any2md
+> in long-running processes should call `any2md.release_models()` or
+> use `with any2md.docling_session():` — see
+> `docs/troubleshooting.md`.
+
 ## The output format
 
-Every file converted by any2md has the same frontmatter shape, regardless of source format. The shape is **SSRM-compatible** — it matches the field names, types, and ordering of the Structured Security Reasoning Markdown specification, but values aren't required to match SSRM's controlled vocabularies (most converted documents aren't security research). Fields that require a controlled vocabulary (`document_type`, `content_domain`, `tlp`) are emitted empty unless you supply them via `--meta` or `.any2md.toml`.
+Every file converted by any2md has the same frontmatter shape, regardless of source format. The shape is **SAGE-compatible** — it matches the field names, types, and ordering of the Security Analysis and Guidance Exchange specification, but values aren't required to match SAGE's controlled vocabularies (most converted documents aren't security research). Fields that require a controlled vocabulary (`document_type`, `content_domain`, `tlp`) are emitted empty unless you supply them via `--meta` or `.any2md.toml`.
 
 ### Field auto-fill summary
 
@@ -404,7 +418,7 @@ You'll see the OK line as usual. The frontmatter of `Text/paper.md` will have `o
 
 ### `.any2md.toml`
 
-You'd use this when you have a stable set of frontmatter defaults that should apply to every conversion in a project. The file is auto-discovered by walking up from the current working directory, so you can drop one at the root of your corpus repo and forget about it. Worked example for a security-research org producing SSRM-conforming outputs:
+You'd use this when you have a stable set of frontmatter defaults that should apply to every conversion in a project. The file is auto-discovered by walking up from the current working directory, so you can drop one at the root of your corpus repo and forget about it. Worked example for a security-research org producing SAGE-conforming outputs:
 
 ```toml
 [meta]
@@ -558,7 +572,7 @@ The two-lane design exists because Docling output is layout-correct (running agg
 
 ## Migrating from v0.7
 
-v1.0 emits SSRM-compatible frontmatter — that's the one intended breaking change. v0.7's small frontmatter (`title`, `source_file`, `pages`, `type`, `word_count`) becomes a fuller block with `document_id`, `version`, `date`, `status`, `content_hash`, `token_estimate`, `authors`, and several other fields. The v0.7 fields are retained as any2md extension fields in the same block, so traceability is preserved, but downstream parsers that expected the old fixed shape will need to update. The body is now NFC-normalized with LF line endings, which means `content_hash` is reproducible. If you need bit-for-bit v0.7 output, pin `any2md==0.7.0`. The full field-by-field migration guide and behavior-change list lives in [docs/upgrading-from-0.7.md](docs/upgrading-from-0.7.md).
+v1.0 emits SAGE-compatible frontmatter — that's the one intended breaking change. v0.7's small frontmatter (`title`, `source_file`, `pages`, `type`, `word_count`) becomes a fuller block with `document_id`, `version`, `date`, `status`, `content_hash`, `token_estimate`, `authors`, and several other fields. The v0.7 fields are retained as any2md extension fields in the same block, so traceability is preserved, but downstream parsers that expected the old fixed shape will need to update. The body is now NFC-normalized with LF line endings, which means `content_hash` is reproducible. If you need bit-for-bit v0.7 output, pin `any2md==0.7.0`. The full field-by-field migration guide and behavior-change list lives in [docs/upgrading-from-0.7.md](docs/upgrading-from-0.7.md).
 
 ## Security
 

{any2md-1.0.5 → any2md-1.1.1}/README.md

@@ -19,7 +19,7 @@ The full design rationale lives in [docs/superpowers/specs/2026-04-26-v1.0.3-emp
 
 ## What this is
 
-any2md ingests heterogeneous source documents and emits Markdown with a fixed YAML frontmatter contract. The frontmatter shape is SSRM-compatible (Structured Security Reasoning Markdown — a documented schema for LLM-consumable documents), so retrieval pipelines, embeddings jobs, and chunking utilities downstream can rely on field names, types, and a deterministic `content_hash` for cache invalidation. The body is NFC-normalized with LF line endings, and the heading hierarchy is guaranteed to start at H1 and not skip levels. The goal is one stable shape, regardless of whether a document started life as a scanned PDF, a Word doc, a Wikipedia page, or a plain text dump.
+any2md ingests heterogeneous source documents and emits Markdown with a fixed YAML frontmatter contract. The frontmatter shape is SAGE-compatible (Security Analysis and Guidance Exchange — a documented schema for LLM-consumable documents), so retrieval pipelines, embeddings jobs, and chunking utilities downstream can rely on field names, types, and a deterministic `content_hash` for cache invalidation. The body is NFC-normalized with LF line endings, and the heading hierarchy is guaranteed to start at H1 and not skip levels. The goal is one stable shape, regardless of whether a document started life as a scanned PDF, a Word doc, a Wikipedia page, or a plain text dump.
 
 ## Quick start
 
@@ -111,8 +111,8 @@ document_id: ""                                    # empty unless you pass --aut
 version: "1"                                       # "1" for source files without an embedded version
 date: "2026-03-15"                                 # docx core props "modified" → ISO-8601
 status: "draft"                                    # always "draft" for converted documents
-document_type: ""                                  # SSRM controlled vocab — empty for non-security docs
-content_domain: []                                 # SSRM controlled vocab — empty array when unknown
+document_type: ""                                  # SAGE controlled vocab — empty for non-security docs
+content_domain: []                                 # SAGE controlled vocab — empty array when unknown
 authors:                                           # extracted from docProps/core.xml dc:creator
   - "Rock Lambros"
 organization: ""                                   # docProps/app.xml Company; empty when absent
@@ -133,7 +133,7 @@ produced_by: "Microsoft® Word for Microsoft 365"   # software that produced the
 ---
 ```
 
-Fields under `# any2md extension field` are retained from v0.7 for traceability and observability — they're not part of the SSRM contract proper, but they live in the same frontmatter block. See [docs/output-format.md](docs/output-format.md) for the full field reference.
+Fields under `# any2md extension field` are retained from v0.7 for traceability and observability — they're not part of the SAGE contract proper, but they live in the same frontmatter block. See [docs/output-format.md](docs/output-format.md) for the full field reference.
 
 ### Before / after
 
@@ -327,9 +327,18 @@ any2md -f -r ./corpus/
 
 You'll see the OK lines as if the conversion were fresh; previously-written outputs are overwritten.
 
+> **v1.1.0+:** when processing PDFs or DOCX files via the Docling
+> backend, the converter (and its loaded model weights) is constructed
+> once on the first PDF/DOCX in an invocation and reused for all
+> subsequent files; subsequent files extract in ~0.5–1s rather than
+> re-paying the ~2.5s cold-load tax. Library users embedding any2md
+> in long-running processes should call `any2md.release_models()` or
+> use `with any2md.docling_session():` — see
+> `docs/troubleshooting.md`.
+
 ## The output format
 
-Every file converted by any2md has the same frontmatter shape, regardless of source format. The shape is **SSRM-compatible** — it matches the field names, types, and ordering of the Structured Security Reasoning Markdown specification, but values aren't required to match SSRM's controlled vocabularies (most converted documents aren't security research). Fields that require a controlled vocabulary (`document_type`, `content_domain`, `tlp`) are emitted empty unless you supply them via `--meta` or `.any2md.toml`.
+Every file converted by any2md has the same frontmatter shape, regardless of source format. The shape is **SAGE-compatible** — it matches the field names, types, and ordering of the Security Analysis and Guidance Exchange specification, but values aren't required to match SAGE's controlled vocabularies (most converted documents aren't security research). Fields that require a controlled vocabulary (`document_type`, `content_domain`, `tlp`) are emitted empty unless you supply them via `--meta` or `.any2md.toml`.
 
 ### Field auto-fill summary
 
@@ -374,7 +383,7 @@ You'll see the OK line as usual. The frontmatter of `Text/paper.md` will have `o
 
 ### `.any2md.toml`
 
-You'd use this when you have a stable set of frontmatter defaults that should apply to every conversion in a project. The file is auto-discovered by walking up from the current working directory, so you can drop one at the root of your corpus repo and forget about it. Worked example for a security-research org producing SSRM-conforming outputs:
+You'd use this when you have a stable set of frontmatter defaults that should apply to every conversion in a project. The file is auto-discovered by walking up from the current working directory, so you can drop one at the root of your corpus repo and forget about it. Worked example for a security-research org producing SAGE-conforming outputs:
 
 ```toml
 [meta]
@@ -528,7 +537,7 @@ The two-lane design exists because Docling output is layout-correct (running agg
 
 ## Migrating from v0.7
 
-v1.0 emits SSRM-compatible frontmatter — that's the one intended breaking change. v0.7's small frontmatter (`title`, `source_file`, `pages`, `type`, `word_count`) becomes a fuller block with `document_id`, `version`, `date`, `status`, `content_hash`, `token_estimate`, `authors`, and several other fields. The v0.7 fields are retained as any2md extension fields in the same block, so traceability is preserved, but downstream parsers that expected the old fixed shape will need to update. The body is now NFC-normalized with LF line endings, which means `content_hash` is reproducible. If you need bit-for-bit v0.7 output, pin `any2md==0.7.0`. The full field-by-field migration guide and behavior-change list lives in [docs/upgrading-from-0.7.md](docs/upgrading-from-0.7.md).
+v1.0 emits SAGE-compatible frontmatter — that's the one intended breaking change. v0.7's small frontmatter (`title`, `source_file`, `pages`, `type`, `word_count`) becomes a fuller block with `document_id`, `version`, `date`, `status`, `content_hash`, `token_estimate`, `authors`, and several other fields. The v0.7 fields are retained as any2md extension fields in the same block, so traceability is preserved, but downstream parsers that expected the old fixed shape will need to update. The body is now NFC-normalized with LF line endings, which means `content_hash` is reproducible. If you need bit-for-bit v0.7 output, pin `any2md==0.7.0`. The full field-by-field migration guide and behavior-change list lives in [docs/upgrading-from-0.7.md](docs/upgrading-from-0.7.md).
 
 ## Security
 

{any2md-1.0.5 → any2md-1.1.1}/SECURITY.md

@@ -81,3 +81,25 @@ If you operate any2md in a hardened environment:
 
 No publicly disclosed vulnerabilities to date. Future advisories will
 appear at https://github.com/rocklambros/any2md/security/advisories.
+
+## Trust boundaries
+
+`any2md` is an operator-trusted CLI distributed via PyPI for desktop and CI use.
+
+**Trusted:** the operator, their filesystem, their `--output-dir`, and the Python interpreter they invoke. CLI flags and `--meta key=val` arguments are operator-controlled.
+
+**Untrusted:**
+- URLs passed to `any2md` (may resolve to internal IPs, redirect, or carry credentials in userinfo/query).
+- Documents (PDF/DOCX/HTML) passed for conversion (may embed crafted images, control characters, malicious metadata).
+- `.any2md.toml` config files discovered via cwd-walking (an attacker who can drop a file in any ancestor directory can inject frontmatter overrides).
+- HTTP responses from URLs (may carry hostile redirects, oversized bodies, control characters in headers).
+
+**Out of scope:** multi-tenant service hosting; running `any2md` against attacker-supplied output directories; supply-chain compromise of pinned dependencies; protection against an operator who has chosen `--output-dir` pointing at a shared/symlinked directory.
+
+## Dependency-drift policy
+
+If `pip-audit` reports a NEW HIGH/CRITICAL vulnerability between baseline and release-cut, the policy is:
+
+1. Bump the dep within its compatible range (regenerate `.devcontainer/requirements.lock` and `requirements.txt`) and re-cut the rc.
+2. If no compatible fix exists, document in CHANGELOG `## Known issues` and ship only if the codepath is mitigated by other code (e.g., XXE blocked by `defusedxml`).
+3. If neither is possible, hold the release until upstream lands a fix.

any2md-1.1.1/any2md/__init__.py

@@ -0,0 +1,11 @@
+"""any2md — convert documents to LLM-friendly Markdown."""
+
+from any2md._docling_cache import docling_session, release_models
+
+__version__ = "1.1.1"
+
+__all__ = [
+    "__version__",
+    "docling_session",
+    "release_models",
+]

{any2md-1.0.5 → any2md-1.1.1}/any2md/_docling.py

@@ -33,5 +33,5 @@ def install_hint() -> None:
     global _hint_emitted
     if _hint_emitted:
         return
-    print(f"  WARN: {INSTALL_HINT_MSG}", file=sys.stderr)
+    print(f"  WARN: {INSTALL_HINT_MSG}", file=sys.stderr)  # log-safe: literal
     _hint_emitted = True