hyperresearch 0.2.0__tar.gz → 0.3.0__tar.gz

{hyperresearch-0.2.0 → hyperresearch-0.3.0}/.gitignore

@@ -11,3 +11,5 @@ build/
 .mypy_cache/
 .ruff_cache/
 *.egg
+.env
+bench/

{hyperresearch-0.2.0 → hyperresearch-0.3.0}/CHANGELOG.md

@@ -1,5 +1,35 @@
 # Changelog
 
+## [0.3.0] - 2026-04-11
+
+### New
+
+- **Native PDF extraction** — PDFs detected by URL pattern, downloaded directly with httpx, text extracted with pymupdf. No browser needed. arXiv `/abs/` links auto-convert to `/pdf/`.
+- **Raw file storage** — PDF bytes saved to `research/raw/<note-id>.pdf`, linked from note frontmatter via `raw_file:` field. Agent can read the raw PDF directly.
+- **Junk page detection** — `WebResult.looks_like_junk()` catches Cloudflare captchas, error pages, cookie walls, binary garbage, reCAPTCHA, and empty content before saving. Returns `JUNK_CONTENT` error instead of creating useless notes.
+- **Gap analysis step** — after drafting the report, agent re-reads the original query word by word, identifies gaps, and does another full round of research to fill them.
+- **Adversarial audit** — two subagents (comprehensiveness auditor + logic/structure auditor) review the draft in parallel. Runs up to 2 loops. Agent uses wait time productively to improve summaries and tags.
+- **Source checkpoint** — agent must review collected sources before writing any draft. Checks coverage breadth, missing angles, uncited references. Expects 50-100+ sources on complex topics.
+- **Scholarly API guidance** — CLAUDE.md and `/research` skill now encourage use of arXiv, Semantic Scholar, CrossRef, and PubMed APIs for academic research.
+- **Date injection** — today's date injected programmatically into CLAUDE.md at install time.
+- **Multi-round research emphasis** — agent docs stress multiple rounds of search → fetch → follow links, spawning 10-20 fetcher agents per round.
+
+### Changed
+
+- **Agent-driven curation replaces auto-enrich** — removed the keyword-matching `enrich_note_file()` from the fetch pipeline. Fetcher subagents now read content, write real summaries, add meaningful tags, and quality-check each source (deprecating junk/off-topic notes).
+- **Fetcher subagent quality gate** — subagent now checks relevance, content quality, and duplicates. Deprecates bad notes instead of leaving them as drafts.
+- **`_resolve_executable()` prioritizes venv** — checks venv `Scripts/` dir before PATH, preventing system-wide installs from overriding the project's venv.
+- **PDF binary detection improved** — checks for `endstream`, `endobj`, `/FlateDecode`, `%PDF-` markers and non-printable character ratios. Catches binary garbage in both single-fetch and batch-fetch paths.
+- **Junk detection thresholds raised** — empty content threshold: 100→300 chars, cookie page threshold: 500→1500 chars. Added `recaptcha`, `checking your browser`, `verify you are human` to bot detection signals.
+- **SSL verification disabled for PDF downloads** — academic sites often have self-signed certs. `httpx.get(verify=False)` for PDF fetches only.
+- **PDF fetch logging** — `_fetch_pdf` failures now logged via `logging.getLogger("hyperresearch.pdf")` instead of silently returning None.
+- **Fetcher subagent continues on failure** — no longer stops on first fetch error, tries all URLs and reports failures individually.
+
+### Added dependencies
+
+- `pymupdf>=1.24` — PDF text extraction
+- `httpx>=0.27` — direct HTTP downloads for PDFs (bypasses browser)
+
 ## [0.2.0] - 2026-04-10
 
 ### New
@@ -68,5 +98,6 @@ Initial release. Forked from [llm-kasten](https://github.com/jordan-gibbs/llm-ka
 - Web viewer with force-directed knowledge graph
 - 70 tests
 
+[0.3.0]: https://github.com/jordan-gibbs/hyperresearch/releases/tag/v0.3.0
 [0.2.0]: https://github.com/jordan-gibbs/hyperresearch/releases/tag/v0.2.0
 [0.1.0]: https://github.com/jordan-gibbs/hyperresearch/releases/tag/v0.1.0

hyperresearch-0.3.0/PKG-INFO

@@ -0,0 +1,188 @@
+Metadata-Version: 2.4
+Name: hyperresearch
+Version: 0.3.0
+Summary: Agent-driven research knowledge base. Browse, collect, and synthesize web sources into a searchable wiki.
+Project-URL: Homepage, https://github.com/jordan-gibbs/hyperresearch
+Project-URL: Repository, https://github.com/jordan-gibbs/hyperresearch
+Project-URL: Issues, https://github.com/jordan-gibbs/hyperresearch/issues
+Author-email: Jordan Gibbs <jordan@jordangibbs.xyz>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: agent,cli,knowledge-base,llm,markdown,research,wiki,zettelkasten
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Text Processing :: Markup :: Markdown
+Requires-Python: >=3.11
+Requires-Dist: crawl4ai>=0.4
+Requires-Dist: httpx>=0.27
+Requires-Dist: jinja2>=3.1
+Requires-Dist: platformdirs>=4.0
+Requires-Dist: pydantic>=2.0
+Requires-Dist: pymupdf>=1.24
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: rich>=13.0
+Requires-Dist: typer>=0.9.0
+Provides-Extra: all
+Requires-Dist: mcp>=1.6; extra == 'all'
+Provides-Extra: dev
+Requires-Dist: mypy>=1.8; extra == 'dev'
+Requires-Dist: pytest-cov>=4.1; extra == 'dev'
+Requires-Dist: pytest>=7.4; extra == 'dev'
+Requires-Dist: ruff>=0.3; extra == 'dev'
+Provides-Extra: mcp
+Requires-Dist: mcp>=1.6; extra == 'mcp'
+Description-Content-Type: text/markdown
+
+<p align="center">
+  <img src="assets/banner.png" alt="HYPERRESEARCH" width="700">
+</p>
+
+<h3 align="center">Ultra deep, persistent web research for AI coding agents</h3>
+
+<p align="center">
+  <a href="https://pypi.org/project/hyperresearch/"><img src="https://img.shields.io/pypi/v/hyperresearch" alt="PyPI version"></a>
+  <a href="https://pypi.org/project/hyperresearch/"><img src="https://img.shields.io/pypi/pyversions/hyperresearch" alt="Python 3.11+"></a>
+  <a href="LICENSE"><img src="https://img.shields.io/github/license/jordan-gibbs/hyperresearch" alt="License: MIT"></a>
+  <a href="https://github.com/jordan-gibbs/hyperresearch"><img src="https://img.shields.io/github/stars/jordan-gibbs/hyperresearch?style=social" alt="GitHub stars"></a>
+</p>
+
+---
+
+Your AI agent searches the web, finds great sources, synthesizes an answer — then the session ends and everything is gone. Next time, it starts from zero.
+
+**Hyperresearch makes research persist.** Every source your agent finds is fetched with a real headless browser, saved as searchable markdown, and indexed into a knowledge base that compounds across sessions. Hyperresearch can follow maps of links to digest paths of information that current agent web searching isn't capable of.
+
+```bash
+pip install hyperresearch
+hyperresearch install
+```
+
+## What people use it for
+
+- **In-depth topic research** — "Research the latest advances in state space models" → agent fetches 20+ papers, docs, and blog posts, follows citations to primary sources, builds a linked knowledge graph
+- **News tracking over time** — run research sessions weekly on a topic, each session builds on what's already collected, nothing gets re-fetched
+- **State-of-the-art surveys** — "What's the current SOTA for speech recognition?" → agent goes down rabbit holes, collects benchmarks, papers, and implementations
+- **Competitive analysis** — scrape company pages, LinkedIn profiles, product docs, news articles into a persistent, searchable corpus
+- **Due diligence** — aggregate everything about a person, company, or technology from across the web with authenticated crawling (LinkedIn, Twitter, paywalled sites)
+
+## How it works
+
+1. **You ask your agent to research something**
+2. **Agent searches the web** — multiple queries, different angles
+3. **Fetches every source** with a real headless browser (crawl4ai) — JS rendering, bot detection bypass, login-gated content
+4. **Saves each page** as a searchable markdown note with tags, summary, and source tracking
+5. **Follows links** to primary sources — the paper, not the blog post about the paper
+6. **Auto-links related notes** with `[[wiki-links]]` across the knowledge graph
+7. **Synthesizes findings** into a summary note linking all sources
+8. **Next session** — agent checks the KB before searching the web. Knowledge compounds.
+
+```
+your-repo/
+  .hyperresearch/        # Config + SQLite FTS5 index (rebuildable)
+  research/
+    notes/               # Markdown notes — the source of truth
+    index/               # Auto-generated wiki pages
+  CLAUDE.md              # Agent docs (auto-injected)
+```
+
+## Works with every major agent
+
+`hyperresearch install` hooks into your agent in one step:
+
+| Platform | Hook | Trigger |
+|----------|------|---------|
+| **Claude Code** | `.claude/settings.json` + `/research` skill | Before WebSearch, WebFetch |
+| **Codex** | `.codex/hooks.json` | Before Bash |
+| **Cursor** | `.cursor/rules/hyperresearch.mdc` | Always-apply rule |
+| **Gemini CLI** | `.gemini/settings.json` | Before tool calls |
+
+```bash
+hyperresearch install --platform all    # Hook every platform at once
+```
+
+## Key features
+
+- **Real headless browser** — crawl4ai runs local Chromium. Handles JavaScript, bypasses bot detection, renders SPAs. Not a simple HTTP fetch.
+- **Native PDF extraction** — fetches PDFs directly, extracts full text with pymupdf, saves raw files to `research/raw/`. Academic papers, whitepapers, reports — all first-class citizens.
+- **Authenticated crawling** — log into LinkedIn, Twitter, paywalled news. Your sessions persist across fetches.
+- **Agent-driven curation** — fetcher subagents read each source, write real summaries, add meaningful tags, and flag junk. No keyword-matching shortcuts.
+- **Junk detection** — Cloudflare captchas, error pages, cookie walls, binary garbage, login redirects all caught and rejected before saving.
+- **Multi-round deep research** — the agent does multiple rounds of search → fetch → follow links, with a source checkpoint that forces breadth before synthesis.
+- **Gap analysis + adversarial audit** — after drafting, the agent re-reads the original query to find gaps, then spawns two auditor subagents (comprehensiveness + logic) to tear the draft apart. Runs twice.
+- **Cheap parallel fetching** — ships a Haiku-powered subagent that fetches, summarizes, and quality-checks URLs in parallel for pennies. Spawn 10-20 per round.
+- **Scholarly API guidance** — agent docs encourage use of arXiv, Semantic Scholar, CrossRef, and PubMed APIs for academic topics.
+- **`/research` skill** — scripted deep research workflow. Clarifies ambiguous requests, searches broadly, fetches aggressively, follows rabbit holes, audits, synthesizes.
+- **Smart SPA wait** — polls DOM stability instead of fixed delays. Fast pages finish instantly, SPAs get up to 10 seconds.
+- **FTS5 search** — instant full-text search across thousands of notes with BM25 ranking
+- **Knowledge graph** — `[[wiki-links]]`, backlinks, hub detection, auto-linking
+- **MCP server** — 13 tools (read + write) for Claude Desktop, Cursor, or any MCP client
+- **Note lifecycle** — draft → review → evergreen → stale → deprecated → archive
+
+## Commands
+
+```bash
+# Research
+hyperresearch fetch <url> --tag t -j           # Fetch a URL into the KB
+hyperresearch fetch-batch <urls...> -j         # Fetch many URLs at once
+hyperresearch research "topic" --max 5 -j      # Full pipeline: search → fetch → link → synthesize
+
+# Search
+hyperresearch search "query" -j                # Full-text search
+hyperresearch search "query" --max-tokens 8000 # Stay within context budget
+hyperresearch note show <id> -j                # Read a note
+
+# Knowledge graph
+hyperresearch link --auto -j                   # Auto-link related notes
+hyperresearch graph hubs -j                    # Most-connected notes
+hyperresearch graph backlinks <id> -j          # What links to this note
+
+# Manage
+hyperresearch sources list -j                  # Every URL ever fetched
+hyperresearch lint -j                          # Health check
+hyperresearch repair -j                        # Fix links, rebuild indexes
+```
+
+Every command returns `{"ok": true, "data": {...}}` with `-j`.
+
+## Authenticated crawling
+
+Fetch from LinkedIn, Twitter, paywalled sites — anything you can log into:
+
+```bash
+hyperresearch setup       # Browser opens. Log into your sites. Done.
+```
+
+```toml
+# .hyperresearch/config.toml
+[web]
+provider = "crawl4ai"
+profile = "research"
+```
+
+LinkedIn, Twitter, Facebook, Instagram, and TikTok automatically use a visible browser to avoid session kills.
+
+## Philosophy
+
+- **No LLM calls.** Hyperresearch stores, indexes, and searches. Your agent is the LLM.
+- **Markdown is truth.** Notes are plain files. SQLite is a rebuildable cache.
+- **Over-collect, then prune.** Fetch aggressively. Deprecate what you don't need.
+- **Check before you fetch.** Hooks kill redundant searches across sessions.
+- **Raw content is king.** Save the original with formatting, not a rewritten summary.
+
+## Requirements
+
+- Python 3.11+
+- Windows, macOS, Linux
+
+## License
+
+[MIT](LICENSE)
+
+## Star History
+
+[![Star History Chart](https://api.star-history.com/svg?repos=jordan-gibbs/hyperresearch&type=Date)](https://star-history.com/#jordan-gibbs/hyperresearch&Date)

hyperresearch-0.3.0/README.md

@@ -0,0 +1,148 @@
+<p align="center">
+  <img src="assets/banner.png" alt="HYPERRESEARCH" width="700">
+</p>
+
+<h3 align="center">Ultra deep, persistent web research for AI coding agents</h3>
+
+<p align="center">
+  <a href="https://pypi.org/project/hyperresearch/"><img src="https://img.shields.io/pypi/v/hyperresearch" alt="PyPI version"></a>
+  <a href="https://pypi.org/project/hyperresearch/"><img src="https://img.shields.io/pypi/pyversions/hyperresearch" alt="Python 3.11+"></a>
+  <a href="LICENSE"><img src="https://img.shields.io/github/license/jordan-gibbs/hyperresearch" alt="License: MIT"></a>
+  <a href="https://github.com/jordan-gibbs/hyperresearch"><img src="https://img.shields.io/github/stars/jordan-gibbs/hyperresearch?style=social" alt="GitHub stars"></a>
+</p>
+
+---
+
+Your AI agent searches the web, finds great sources, synthesizes an answer — then the session ends and everything is gone. Next time, it starts from zero.
+
+**Hyperresearch makes research persist.** Every source your agent finds is fetched with a real headless browser, saved as searchable markdown, and indexed into a knowledge base that compounds across sessions. Hyperresearch can follow maps of links to digest paths of information that current agent web searching isn't capable of.
+
+```bash
+pip install hyperresearch
+hyperresearch install
+```
+
+## What people use it for
+
+- **In-depth topic research** — "Research the latest advances in state space models" → agent fetches 20+ papers, docs, and blog posts, follows citations to primary sources, builds a linked knowledge graph
+- **News tracking over time** — run research sessions weekly on a topic, each session builds on what's already collected, nothing gets re-fetched
+- **State-of-the-art surveys** — "What's the current SOTA for speech recognition?" → agent goes down rabbit holes, collects benchmarks, papers, and implementations
+- **Competitive analysis** — scrape company pages, LinkedIn profiles, product docs, news articles into a persistent, searchable corpus
+- **Due diligence** — aggregate everything about a person, company, or technology from across the web with authenticated crawling (LinkedIn, Twitter, paywalled sites)
+
+## How it works
+
+1. **You ask your agent to research something**
+2. **Agent searches the web** — multiple queries, different angles
+3. **Fetches every source** with a real headless browser (crawl4ai) — JS rendering, bot detection bypass, login-gated content
+4. **Saves each page** as a searchable markdown note with tags, summary, and source tracking
+5. **Follows links** to primary sources — the paper, not the blog post about the paper
+6. **Auto-links related notes** with `[[wiki-links]]` across the knowledge graph
+7. **Synthesizes findings** into a summary note linking all sources
+8. **Next session** — agent checks the KB before searching the web. Knowledge compounds.
+
+```
+your-repo/
+  .hyperresearch/        # Config + SQLite FTS5 index (rebuildable)
+  research/
+    notes/               # Markdown notes — the source of truth
+    index/               # Auto-generated wiki pages
+  CLAUDE.md              # Agent docs (auto-injected)
+```
+
+## Works with every major agent
+
+`hyperresearch install` hooks into your agent in one step:
+
+| Platform | Hook | Trigger |
+|----------|------|---------|
+| **Claude Code** | `.claude/settings.json` + `/research` skill | Before WebSearch, WebFetch |
+| **Codex** | `.codex/hooks.json` | Before Bash |
+| **Cursor** | `.cursor/rules/hyperresearch.mdc` | Always-apply rule |
+| **Gemini CLI** | `.gemini/settings.json` | Before tool calls |
+
+```bash
+hyperresearch install --platform all    # Hook every platform at once
+```
+
+## Key features
+
+- **Real headless browser** — crawl4ai runs local Chromium. Handles JavaScript, bypasses bot detection, renders SPAs. Not a simple HTTP fetch.
+- **Native PDF extraction** — fetches PDFs directly, extracts full text with pymupdf, saves raw files to `research/raw/`. Academic papers, whitepapers, reports — all first-class citizens.
+- **Authenticated crawling** — log into LinkedIn, Twitter, paywalled news. Your sessions persist across fetches.
+- **Agent-driven curation** — fetcher subagents read each source, write real summaries, add meaningful tags, and flag junk. No keyword-matching shortcuts.
+- **Junk detection** — Cloudflare captchas, error pages, cookie walls, binary garbage, login redirects all caught and rejected before saving.
+- **Multi-round deep research** — the agent does multiple rounds of search → fetch → follow links, with a source checkpoint that forces breadth before synthesis.
+- **Gap analysis + adversarial audit** — after drafting, the agent re-reads the original query to find gaps, then spawns two auditor subagents (comprehensiveness + logic) to tear the draft apart. Runs twice.
+- **Cheap parallel fetching** — ships a Haiku-powered subagent that fetches, summarizes, and quality-checks URLs in parallel for pennies. Spawn 10-20 per round.
+- **Scholarly API guidance** — agent docs encourage use of arXiv, Semantic Scholar, CrossRef, and PubMed APIs for academic topics.
+- **`/research` skill** — scripted deep research workflow. Clarifies ambiguous requests, searches broadly, fetches aggressively, follows rabbit holes, audits, synthesizes.
+- **Smart SPA wait** — polls DOM stability instead of fixed delays. Fast pages finish instantly, SPAs get up to 10 seconds.
+- **FTS5 search** — instant full-text search across thousands of notes with BM25 ranking
+- **Knowledge graph** — `[[wiki-links]]`, backlinks, hub detection, auto-linking
+- **MCP server** — 13 tools (read + write) for Claude Desktop, Cursor, or any MCP client
+- **Note lifecycle** — draft → review → evergreen → stale → deprecated → archive
+
+## Commands
+
+```bash
+# Research
+hyperresearch fetch <url> --tag t -j           # Fetch a URL into the KB
+hyperresearch fetch-batch <urls...> -j         # Fetch many URLs at once
+hyperresearch research "topic" --max 5 -j      # Full pipeline: search → fetch → link → synthesize
+
+# Search
+hyperresearch search "query" -j                # Full-text search
+hyperresearch search "query" --max-tokens 8000 # Stay within context budget
+hyperresearch note show <id> -j                # Read a note
+
+# Knowledge graph
+hyperresearch link --auto -j                   # Auto-link related notes
+hyperresearch graph hubs -j                    # Most-connected notes
+hyperresearch graph backlinks <id> -j          # What links to this note
+
+# Manage
+hyperresearch sources list -j                  # Every URL ever fetched
+hyperresearch lint -j                          # Health check
+hyperresearch repair -j                        # Fix links, rebuild indexes
+```
+
+Every command returns `{"ok": true, "data": {...}}` with `-j`.
+
+## Authenticated crawling
+
+Fetch from LinkedIn, Twitter, paywalled sites — anything you can log into:
+
+```bash
+hyperresearch setup       # Browser opens. Log into your sites. Done.
+```
+
+```toml
+# .hyperresearch/config.toml
+[web]
+provider = "crawl4ai"
+profile = "research"
+```
+
+LinkedIn, Twitter, Facebook, Instagram, and TikTok automatically use a visible browser to avoid session kills.
+
+## Philosophy
+
+- **No LLM calls.** Hyperresearch stores, indexes, and searches. Your agent is the LLM.
+- **Markdown is truth.** Notes are plain files. SQLite is a rebuildable cache.
+- **Over-collect, then prune.** Fetch aggressively. Deprecate what you don't need.
+- **Check before you fetch.** Hooks kill redundant searches across sessions.
+- **Raw content is king.** Save the original with formatting, not a rewritten summary.
+
+## Requirements
+
+- Python 3.11+
+- Windows, macOS, Linux
+
+## License
+
+[MIT](LICENSE)
+
+## Star History
+
+[![Star History Chart](https://api.star-history.com/svg?repos=jordan-gibbs/hyperresearch&type=Date)](https://star-history.com/#jordan-gibbs/hyperresearch&Date)

{hyperresearch-0.2.0 → hyperresearch-0.3.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "hyperresearch"
-version = "0.2.0"
+version = "0.3.0"
 description = "Agent-driven research knowledge base. Browse, collect, and synthesize web sources into a searchable wiki."
 readme = "README.md"
 license = "MIT"
@@ -28,12 +28,14 @@ dependencies = [
     "pydantic>=2.0",
     "jinja2>=3.1",
     "platformdirs>=4.0",
+    "Crawl4AI>=0.4",
+    "pymupdf>=1.24",
+    "httpx>=0.27",
 ]
 
 [project.optional-dependencies]
-crawl4ai = ["Crawl4AI>=0.4"]
 mcp = ["mcp>=1.6"]
-all = ["hyperresearch[crawl4ai,mcp]"]
+all = ["hyperresearch[mcp]"]
 dev = [
     "pytest>=7.4",
     "pytest-cov>=4.1",

{hyperresearch-0.2.0 → hyperresearch-0.3.0}/src/hyperresearch/cli/fetch.py

@@ -116,6 +116,16 @@ def fetch(
             console.print(f"[red]Auth required:[/] {msg}")
         raise typer.Exit(1)
 
+    # Detect junk pages — captcha, error pages, binary garbage, empty content
+    junk_reason = result.looks_like_junk()
+    if junk_reason:
+        msg = f"Skipped junk content from {url}: {junk_reason}"
+        if json_output:
+            output(error(msg, "JUNK_CONTENT"), json_mode=True)
+        else:
+            console.print(f"[yellow]Skipped:[/] {msg}")
+        raise typer.Exit(1)
+
     # Write note
     note_title = title or result.title or urlparse(url).path.split("/")[-1] or "Untitled"
     domain = result.domain
@@ -140,10 +150,40 @@ def fetch(
         extra_frontmatter=extra_meta,
     )
 
-    # Auto-enrich: add suggested tags and summary before sync
-    from hyperresearch.core.enrich import enrich_note_file
-
-    enrich_note_file(note_path, conn, tags)
+    # Save raw file (PDF, etc.) if present
+    raw_file_path = None
+    if result.raw_bytes and result.raw_content_type:
+        ext_map = {
+            "application/pdf": ".pdf",
+            "image/png": ".png",
+            "image/jpeg": ".jpg",
+            "image/gif": ".gif",
+            "image/webp": ".webp",
+        }
+        ext = ext_map.get(result.raw_content_type, "")
+        if ext:
+            raw_dir = vault.root / "research" / "raw"
+            raw_dir.mkdir(parents=True, exist_ok=True)
+            raw_filename = note_path.stem + ext
+            raw_file = raw_dir / raw_filename
+            raw_file.write_bytes(result.raw_bytes)
+            raw_file_path = f"raw/{raw_filename}"
+
+    # Note: tagging and summarization is the agent's job, not an automatic process.
+    # The agent reads the fetched content and writes meaningful summaries and tags.
+
+    # Add raw_file reference to frontmatter AFTER enrich (enrich rewrites frontmatter)
+    if raw_file_path:
+        note_text = note_path.read_text(encoding="utf-8")
+        if note_text.startswith("---") and "raw_file:" not in note_text:
+            end_idx = note_text.find("---", 3)
+            if end_idx != -1:
+                note_text = (
+                    note_text[:end_idx]
+                    + f"raw_file: {raw_file_path}\n"
+                    + note_text[end_idx:]
+                )
+                note_path.write_text(note_text, encoding="utf-8")
 
     # Sync first so the note exists in the notes table (needed for FK on sources/assets)
     note_id = note_path.stem
@@ -175,6 +215,7 @@ def fetch(
         "path": str(note_path.relative_to(vault.root)),
         "word_count": len(result.content.split()),
         "assets": saved_assets,
+        "raw_file": raw_file_path,
     }
 
     if json_output: