gather-engine 2.0.0__tar.gz → 2.2.0__tar.gz

{gather_engine-2.0.0 → gather_engine-2.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: gather-engine
-Version: 2.0.0
+Version: 2.2.0
 Summary: Accountable research intake for difficult sources, with provenance receipts and witnessed digests.
 Author: Zain Dana Harper
 License-Expression: LicenseRef-FSL-1.1-MIT
@@ -34,7 +34,7 @@ Dynamic: license-file
 [![downloads](https://img.shields.io/pypi/dm/gather-engine?label=downloads&style=flat-square&labelColor=14041b)](https://pypi.org/project/gather-engine/)
 ![python: 3.11+](https://img.shields.io/badge/python-3.11%2B-blue?style=flat-square&labelColor=14041b)
 ![deps: none (core)](https://img.shields.io/badge/core%20deps-none-success?style=flat-square&labelColor=14041b)
-![version: 2.0.0](https://img.shields.io/badge/version-2.0.0-26dfe8?style=flat-square&labelColor=14041b)
+![version: 2.2.0](https://img.shields.io/badge/version-2.2.0-26dfe8?style=flat-square&labelColor=14041b)
 
 gather pulls research out of the places most tools break on: arXiv papers, authenticated JSON APIs, JavaScript-rendered pages via a real headless browser, scanned images through OCR, and audio through transcription, alongside video, web, feeds, and local docs. The core runs with zero third-party runtime dependencies, and the same engine is reachable from the CLI, MCP tools, and plain Python. Every run writes a receipt you can re-check.
 
@@ -42,7 +42,7 @@ gather pulls research out of the places most tools break on: arXiv papers, authe
 
 ## Current status
 
-`gather-engine 2.0.0` is the current source version. Core intake,
+`gather-engine 2.2.0` is the current source version. Core intake,
 content-addressed corpus storage, exact UTF-8 source-byte receipts, readable
 context selection, descriptor handoff, pilot monitoring, CLI, Python API, and MCP
 surfaces are present in this checkout. Browser rendering, OCR, audio, and fast
@@ -82,7 +82,9 @@ fabricated field cannot reach the corpus by being plausible.
 - **Track elements across redesigns.** Fingerprint a scraped element once, then relocate it in any later version of the page and get a typed MATCH, RELOCATED, DRIFT, or GONE verdict (`gather.track`, Python API).
 - **Structured extraction with a hallucination check.** `gather.schema_extract` binds schema fields to source nodes, and `verify_record` rejects any LLM-proposed field value not grounded in the fetched content.
 - **Streaming extraction.** `gather.stream` parses an HTML stream chunk by chunk and emits partial-update commits as blocks complete, each folded into a hash chain, so a streamed extraction is replayable.
-- **Hard-source adapters behind one shape.** Video with captions and comments (`yt-dlp`), static web, RSS/Atom feeds, local docs, arXiv, PDFs (`pdftotext`), authenticated JSON APIs (token from env, never logged), JS-rendered pages (headless Chromium), scanned images (`tesseract`), and audio (`whisper`). Each external tool is optional and only needed for its own adapter.
+- **Hard-source adapters behind one shape.** Video with captions and comments (`yt-dlp` first, with the official YouTube Data API as a fallback when you supply your own key; each item records which path served it and how fast), Reddit through its official Data API with your own app credentials, static web, RSS/Atom feeds, local docs, arXiv, PDFs (`pdftotext`), authenticated JSON APIs (token from env, never logged), JS-rendered pages (headless Chromium), scanned images (`tesseract`), and audio (`whisper`). Each external tool is optional and only needed for its own adapter.
+- **Cited reports, checked in code.** `gather report` asks a model on your own machine to answer from fixed excerpts with exact quotes, then checks every quote against the excerpt it cites and marks any it cannot verify.
+- **Filter ledgers.** `--scope TERMS --ledger DIR` writes one row per dropped item with its reason code, plus the input, so `gather ledger verify` can recompute every count.
 - **Channel and playlist intake.** `gather channel <url> --store DIR` lists a channel's videos, shorts, and streams tabs (or one playlist) and gathers every entry with bounded concurrency, resumable from a per-pass ledger. Metadata and comments run as one pass and captions as another (`--no-captions`, `--captions-only`), with polite pacing, bounded backoff on HTTP 429 and session rate limits, a stop at the first bot check (never retried or answered), one caption track per video, and a run summary that counts every outcome by reason.
 - **Scholarly-graph federation.** `gather scholar` queries OpenAlex, Semantic Scholar, and Crossref in one call, dedupes results by normalized DOI (never a fuzzy title match), and can capture citation edges as first-class records with `--edges`.
 - **A durable local corpus.** Any fetch command takes `--store DIR`: bodies are content-addressed and deduped by hash, new writes preserve exact UTF-8 source bytes, and `gather corpus list|verify|digest|runs|search|stats|prune|availability|context` inspects, re-checks, and queries what you stored.
@@ -250,4 +252,15 @@ Keep the README, package metadata, and examples aligned with current behavior be
 
 ---
 
-**[Zentropy Labs](https://github.com/ZentropyLabs-ai)** · order out of entropy. An independent lab building evidence-first tools that leave a re-checkable artifact behind. Built by Zain Dana Harper in Seattle. The full workbench is at [Project Telos](https://harperz9.github.io).
+Built by **[Zain Dana Harper](https://harperz9.github.io)** in Seattle: evidence-first tools that leave a re-checkable artifact behind. The full workbench is at [Project Telos](https://harperz9.github.io).
+
+## Local client packages
+
+The optional client bundle adds a bounded read-only MCP profile with an explicit
+workspace selected at launch. See [client package setup](client-plugin/README.md).
+Network retrieval is disabled until an exact origin is granted at launch.
+Allowed GET requests return source text and byte-hash receipts; redirects,
+ambient proxies, credentials and process execution remain disabled.
+Windows x64 MCPB and ZIP bundles include their runtime; source plugins require
+Python 3.11+. The full CLI/MCP retains advanced operations. Client-specific
+installation and marketplace acceptance require separate verification.

{gather_engine-2.0.0 → gather_engine-2.2.0}/README.md

@@ -8,7 +8,7 @@
 [![downloads](https://img.shields.io/pypi/dm/gather-engine?label=downloads&style=flat-square&labelColor=14041b)](https://pypi.org/project/gather-engine/)
 ![python: 3.11+](https://img.shields.io/badge/python-3.11%2B-blue?style=flat-square&labelColor=14041b)
 ![deps: none (core)](https://img.shields.io/badge/core%20deps-none-success?style=flat-square&labelColor=14041b)
-![version: 2.0.0](https://img.shields.io/badge/version-2.0.0-26dfe8?style=flat-square&labelColor=14041b)
+![version: 2.2.0](https://img.shields.io/badge/version-2.2.0-26dfe8?style=flat-square&labelColor=14041b)
 
 gather pulls research out of the places most tools break on: arXiv papers, authenticated JSON APIs, JavaScript-rendered pages via a real headless browser, scanned images through OCR, and audio through transcription, alongside video, web, feeds, and local docs. The core runs with zero third-party runtime dependencies, and the same engine is reachable from the CLI, MCP tools, and plain Python. Every run writes a receipt you can re-check.
 
@@ -16,7 +16,7 @@ gather pulls research out of the places most tools break on: arXiv papers, authe
 
 ## Current status
 
-`gather-engine 2.0.0` is the current source version. Core intake,
+`gather-engine 2.2.0` is the current source version. Core intake,
 content-addressed corpus storage, exact UTF-8 source-byte receipts, readable
 context selection, descriptor handoff, pilot monitoring, CLI, Python API, and MCP
 surfaces are present in this checkout. Browser rendering, OCR, audio, and fast
@@ -56,7 +56,9 @@ fabricated field cannot reach the corpus by being plausible.
 - **Track elements across redesigns.** Fingerprint a scraped element once, then relocate it in any later version of the page and get a typed MATCH, RELOCATED, DRIFT, or GONE verdict (`gather.track`, Python API).
 - **Structured extraction with a hallucination check.** `gather.schema_extract` binds schema fields to source nodes, and `verify_record` rejects any LLM-proposed field value not grounded in the fetched content.
 - **Streaming extraction.** `gather.stream` parses an HTML stream chunk by chunk and emits partial-update commits as blocks complete, each folded into a hash chain, so a streamed extraction is replayable.
-- **Hard-source adapters behind one shape.** Video with captions and comments (`yt-dlp`), static web, RSS/Atom feeds, local docs, arXiv, PDFs (`pdftotext`), authenticated JSON APIs (token from env, never logged), JS-rendered pages (headless Chromium), scanned images (`tesseract`), and audio (`whisper`). Each external tool is optional and only needed for its own adapter.
+- **Hard-source adapters behind one shape.** Video with captions and comments (`yt-dlp` first, with the official YouTube Data API as a fallback when you supply your own key; each item records which path served it and how fast), Reddit through its official Data API with your own app credentials, static web, RSS/Atom feeds, local docs, arXiv, PDFs (`pdftotext`), authenticated JSON APIs (token from env, never logged), JS-rendered pages (headless Chromium), scanned images (`tesseract`), and audio (`whisper`). Each external tool is optional and only needed for its own adapter.
+- **Cited reports, checked in code.** `gather report` asks a model on your own machine to answer from fixed excerpts with exact quotes, then checks every quote against the excerpt it cites and marks any it cannot verify.
+- **Filter ledgers.** `--scope TERMS --ledger DIR` writes one row per dropped item with its reason code, plus the input, so `gather ledger verify` can recompute every count.
 - **Channel and playlist intake.** `gather channel <url> --store DIR` lists a channel's videos, shorts, and streams tabs (or one playlist) and gathers every entry with bounded concurrency, resumable from a per-pass ledger. Metadata and comments run as one pass and captions as another (`--no-captions`, `--captions-only`), with polite pacing, bounded backoff on HTTP 429 and session rate limits, a stop at the first bot check (never retried or answered), one caption track per video, and a run summary that counts every outcome by reason.
 - **Scholarly-graph federation.** `gather scholar` queries OpenAlex, Semantic Scholar, and Crossref in one call, dedupes results by normalized DOI (never a fuzzy title match), and can capture citation edges as first-class records with `--edges`.
 - **A durable local corpus.** Any fetch command takes `--store DIR`: bodies are content-addressed and deduped by hash, new writes preserve exact UTF-8 source bytes, and `gather corpus list|verify|digest|runs|search|stats|prune|availability|context` inspects, re-checks, and queries what you stored.
@@ -224,4 +226,15 @@ Keep the README, package metadata, and examples aligned with current behavior be
 
 ---
 
-**[Zentropy Labs](https://github.com/ZentropyLabs-ai)** · order out of entropy. An independent lab building evidence-first tools that leave a re-checkable artifact behind. Built by Zain Dana Harper in Seattle. The full workbench is at [Project Telos](https://harperz9.github.io).
+Built by **[Zain Dana Harper](https://harperz9.github.io)** in Seattle: evidence-first tools that leave a re-checkable artifact behind. The full workbench is at [Project Telos](https://harperz9.github.io).
+
+## Local client packages
+
+The optional client bundle adds a bounded read-only MCP profile with an explicit
+workspace selected at launch. See [client package setup](client-plugin/README.md).
+Network retrieval is disabled until an exact origin is granted at launch.
+Allowed GET requests return source text and byte-hash receipts; redirects,
+ambient proxies, credentials and process execution remain disabled.
+Windows x64 MCPB and ZIP bundles include their runtime; source plugins require
+Python 3.11+. The full CLI/MCP retains advanced operations. Client-specific
+installation and marketplace acceptance require separate verification.

{gather_engine-2.0.0 → gather_engine-2.2.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "gather-engine"
-version = "2.0.0"
+version = "2.2.0"
 description = "Accountable research intake for difficult sources, with provenance receipts and witnessed digests."
 readme = "README.md"
 requires-python = ">=3.11"

{gather_engine-2.0.0 → gather_engine-2.2.0}/src/gather/__init__.py

@@ -28,7 +28,7 @@ from gather.scope import filter_scope, in_scope
 from gather.source import Catalog, Source
 from gather.store import Corpus
 
-__version__ = "2.0.0"
+__version__ = "2.2.0"
 
 __all__ = [
     "Catalog", "Corpus", "CorpusRootDescriptor", "CorpusRootIdentity", "Digest", "Item", "NullProvenanceProvider", "NullSynthesizer",

gather_engine-2.2.0/src/gather/citecheck.py

@@ -0,0 +1,119 @@
+"""A code check on every citation in a report. Pure: no network, no model.
+
+A report cites an excerpt by quoting it and naming its number: ``"the exact words" [2]``. For
+each citation the check decides, by string comparison alone:
+
+- ``verified``: the quoted span appears in excerpt 2 (after Unicode, quote-mark, dash and
+  whitespace normalisation, and case folding);
+- ``not-in-source``: excerpt 2 exists and does not contain the span;
+- ``unknown-source``: there is no excerpt 2;
+- ``too-short``: the quote has fewer than ``MIN_QUOTE_WORDS`` words, so a match would prove
+  little; it is not counted as verified;
+- ``unchecked``: a bracketed number with no quote before it, so there is nothing to compare.
+
+Sentences with no citation at all are listed as uncited. Nothing is hidden: every citation and
+every uncited sentence is in the result. Precision is verified citations over all citations.
+
+What this does not check: whether the sentence around a verified quote says what the excerpt
+means. A quote can be exact and the claim around it can still stretch it.
+"""
+
+from __future__ import annotations
+
+import re
+import unicodedata
+from dataclasses import dataclass
+
+CHECK_SCHEMA = "gather.citation-check/1"
+MIN_QUOTE_WORDS = 4
+STATUSES = ("verified", "not-in-source", "unknown-source", "too-short", "unchecked")
+
+_MARKS = str.maketrans({"“": '"', "”": '"', "„": '"', "«": '"', "»": '"',
+                        "‘": "'", "’": "'", "–": "-", "—": "-", "−": "-",
+                        " ": " "})
+# a double-quoted span, optional space, then [n]
+_QUOTE_CITE = re.compile(r'"([^"]{1,2000})"\s*\[(\d{1,4})\]')
+_BARE_CITE = re.compile(r"\[(\d{1,4})\]")
+_LIST_MARK = re.compile(r"^(?:[-*+]|\d{1,3}[.)])\s+")
+_SENTENCE = re.compile(r'(?<=[.!?])["\')\]]*\s+(?=["(\[A-Z0-9])')
+
+
+def normalize(text: str) -> str:
+    """NFKC, straight quote marks and hyphens, collapsed whitespace, case folded."""
+    text = unicodedata.normalize("NFKC", text).translate(_MARKS)
+    return re.sub(r"\s+", " ", text).strip().casefold()
+
+
+def _strip_edges(quote: str) -> str:
+    return normalize(quote).strip(" .,;:!?'")
+
+
+@dataclass(frozen=True, slots=True)
+class Citation:
+    sentence: int
+    source: int
+    quote: str | None
+    status: str
+
+    def to_dict(self) -> dict:
+        return {"sentence": self.sentence, "source": self.source, "quote": self.quote, "status": self.status}
+
+
+def _split_line(line: str) -> list[str]:
+    """Split one line at sentence ends that fall outside a double-quoted span."""
+    parts, start = [], 0
+    for m in _SENTENCE.finditer(line):
+        if line[:m.start()].count('"') % 2 == 0:
+            parts.append(line[start:m.start() + len(m.group(0).rstrip())].strip())
+            start = m.end()
+    parts.append(line[start:].strip())
+    return [p for p in parts if p]
+
+
+def sentences(report: str) -> list[str]:
+    """The report's sentences: lines split at sentence ends outside quotes, list markers removed,
+    headings and blank lines dropped."""
+    out: list[str] = []
+    for raw in report.splitlines():
+        line = raw.strip()
+        if not line or line.startswith("#"):
+            continue
+        out.extend(_split_line(_LIST_MARK.sub("", line.translate(_MARKS))))
+    return out
+
+
+def _status(quote: str, source: int, excerpts: list[str]) -> str:
+    if not 1 <= source <= len(excerpts):
+        return "unknown-source"
+    span = _strip_edges(quote)
+    if len(span.split()) < MIN_QUOTE_WORDS:
+        return "too-short"
+    return "verified" if span in normalize(excerpts[source - 1]) else "not-in-source"
+
+
+def _sentence_citations(n: int, sentence: str, excerpts: list[str]) -> list[Citation]:
+    found = [Citation(n, int(m.group(2)), m.group(1), _status(m.group(1), int(m.group(2)), excerpts))
+             for m in _QUOTE_CITE.finditer(sentence.translate(_MARKS))]
+    bare_text = _QUOTE_CITE.sub("", sentence.translate(_MARKS))
+    found += [Citation(n, int(m.group(1)), None, "unchecked") for m in _BARE_CITE.finditer(bare_text)]
+    return found
+
+
+def check_report(report: str, excerpts: list[str]) -> dict:
+    """Check every citation in ``report`` against ``excerpts`` (excerpt 1 is ``excerpts[0]``)."""
+    cites: list[Citation] = []
+    uncited: list[str] = []
+    for n, sentence in enumerate(sentences(report)):
+        found = _sentence_citations(n, sentence, excerpts)
+        cites.extend(found)
+        if not found:
+            uncited.append(sentence)
+    counts = {s: sum(1 for c in cites if c.status == s) for s in STATUSES}
+    total = len(cites)
+    return {
+        "schema": CHECK_SCHEMA, "excerpts": len(excerpts), "citations": [c.to_dict() for c in cites],
+        "counts": counts, "total": total,
+        "precision": round(counts["verified"] / total, 4) if total else None,
+        "uncited_sentences": uncited,
+        "does_not_prove": "that the claim around a verified quote matches the excerpt's meaning",
+    }

{gather_engine-2.0.0 → gather_engine-2.2.0}/src/gather/cli.py

@@ -4,6 +4,7 @@ import argparse
 import sys
 
 from gather import __version__
+from gather.cli_ext import add_extension_parsers
 from gather.commands import (
     cmd_api,
     cmd_arxiv,
@@ -11,12 +12,10 @@ from gather.commands import (
     cmd_docs,
     cmd_feed,
     cmd_ocr,
-    cmd_parse,
     cmd_pdf,
     cmd_run,
     cmd_scholar,
     cmd_transcribe,
-    cmd_video,
     cmd_web,
 )
 from gather.corpus_cmd import cmd_corpus
@@ -29,7 +28,6 @@ from gather.pilot_commands import (
     cmd_pilot_run,
     cmd_pilot_verify,
 )
-from gather.video_cmd import add_channel_parser, add_ytdlp_options
 from gather.web_commands import (
     cmd_caps,
     cmd_crawl,
@@ -50,6 +48,8 @@ def _add_common(p: argparse.ArgumentParser) -> None:
     p.add_argument("--scope", default=None, help="comma-separated scope terms; keep items mentioning any")
     p.add_argument("--json", action="store_true", help="emit the catalog and digest as JSON")
     p.add_argument("--store", default=None, metavar="DIR", help="persist gathered items into a corpus at DIR")
+    p.add_argument("--ledger", default=None, metavar="DIR",
+                   help="write a filter ledger (ledger.json, input.jsonl) of what --scope dropped into DIR")
 
 
 def _add_flagship_commands(sub) -> None:
@@ -124,21 +124,7 @@ def build_parser() -> argparse.ArgumentParser:
     sub = parser.add_subparsers(dest="command")
     _add_flagship_commands(sub)
 
-    parse = sub.add_parser("parse", help="parse a saved yt-dlp info.json (+ optional .vtt), offline, no network")
-    parse.add_argument("info", help="path to a yt-dlp info.json")
-    parse.add_argument("--vtt", default=None, help="path to a .vtt captions file")
-    parse.add_argument("--auto-captions", action="store_true",
-                       help="captions are machine-generated: collapse rolling-window growth, stamp auto-caption")
-    _add_common(parse)
-    parse.set_defaults(func=cmd_parse)
-
-    video = sub.add_parser("video", help="fetch a video via yt-dlp (needs yt-dlp on PATH and network)")
-    video.add_argument("url")
-    video.add_argument("--comments", action="store_true", help="also gather comments")
-    add_ytdlp_options(video)
-    _add_common(video)
-    video.set_defaults(func=cmd_video)
-    add_channel_parser(sub)
+    add_extension_parsers(sub, _add_common)
 
     web = sub.add_parser("web", help="fetch a static web page via http(s) and extract readable text")
     web.add_argument("url")

gather_engine-2.2.0/src/gather/cli_ext.py

@@ -0,0 +1,19 @@
+"""The subcommands that live in their own modules, registered in one place for ``gather.cli``."""
+
+from __future__ import annotations
+
+from collections.abc import Callable
+
+
+def add_extension_parsers(sub, add_common: Callable) -> None:
+    from gather.ledger_cmd import add_ledger_parser
+    from gather.reddit_cmd import add_reddit_parser
+    from gather.report_cmd import add_report_parsers
+    from gather.video_cmd import add_channel_parser
+    from gather.youtube_cmd import add_video_parsers
+
+    add_video_parsers(sub, add_common)
+    add_channel_parser(sub)
+    add_reddit_parser(sub, add_common)
+    add_report_parsers(sub, add_common)
+    add_ledger_parser(sub)

gather_engine-2.2.0/src/gather/client_mcp.py

@@ -0,0 +1,178 @@
+"""Local client profile: launch-root confinement and explicit tool allowlist.
+
+This is a read-only convenience boundary, not an OS sandbox. Concurrent local
+filesystem mutation is outside its threat model. Full CLI/MCP remains separate.
+"""
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import stat
+import sys
+from pathlib import Path
+
+MAX_FILE = 8_000_000
+MAX_ENTRIES = 20_000
+NAME = "gather"
+PACKAGE = "gather"
+
+
+class ClientRefusal(ValueError):
+    def __init__(self, detail, code="PATH_DENIED"):
+        super().__init__(detail)
+        self.code = code
+
+
+def confined(root: Path, value: object, *, tree=False) -> Path:
+    if not isinstance(value, str) or not value or chr(0) in value:
+        raise ClientRefusal("path must be a non-empty string")
+    # Reject Windows network/device/ADS paths before resolving or accessing them.
+    text = value.replace(chr(92), "/")
+    if text.startswith("//") or ":" in text[2:] or (":" in text and not (len(text) > 2 and text[1:3] == ":/")):
+        raise ClientRefusal("network, device and alternate-stream paths are not permitted")
+    candidate = Path(value)
+    if not candidate.is_absolute():
+        candidate = root / candidate
+    path = candidate.resolve(strict=True)
+    if path != root and root not in path.parents:
+        raise ClientRefusal("path is outside the launch workspace")
+    for part in [candidate, *candidate.parents]:
+        if part == root.parent:
+            break
+        info = part.lstat()
+        if stat.S_ISLNK(info.st_mode) or getattr(info, "st_file_attributes", 0) & 0x400:
+            raise ClientRefusal("links and reparse points are not permitted")
+    if path.is_file() and path.stat().st_size > MAX_FILE:
+        raise ClientRefusal("file exceeds local client limit")
+    if tree and path.is_dir():
+        count = 0
+        for folder, dirs, files in os.walk(path, followlinks=False):
+            dirs[:] = [d for d in dirs if d not in {".git", ".venv", "node_modules", "__pycache__"}]
+            for name in [*dirs, *files]:
+                count += 1
+                if count > MAX_ENTRIES:
+                    raise ClientRefusal("workspace exceeds local client entry limit")
+                confined(root, str(Path(folder) / name))
+    return path
+
+
+def handle(req, root, network=None):
+    if not isinstance(req, dict):
+        return {"jsonrpc": "2.0", "id": None, "error": {"code": -32600, "message": "invalid request"}}
+    if "id" not in req:
+        return None
+    response = {"jsonrpc": "2.0", "id": req["id"]}
+    method = req.get("method")
+    if method == "initialize":
+        from gather import __version__
+        response["result"] = {"protocolVersion": "2025-06-18", "capabilities": {"tools": {}},
+                              "serverInfo": {"name": NAME + "-local", "version": __version__}}
+    elif method == "ping":
+        response["result"] = {}
+    elif method == "tools/list":
+        response["result"] = {"tools": definitions(network)}
+    elif method == "tools/call":
+        try:
+            params = req.get("params") or {}
+            args = params.get("arguments") or {}
+            name = params.get("name")
+            definition = next((d for d in definitions(network) if d["name"] == name), None)
+            if definition is None:
+                raise ClientRefusal("tool requires the separately configured full MCP surface", "TOOL_NOT_GRANTED")
+            if not isinstance(args, dict) or set(args) - set(definition["inputSchema"]["properties"]):
+                raise ClientRefusal("unsupported arguments cannot grant permissions", "ARGUMENTS_DENIED")
+            missing = set(definition["inputSchema"].get("required", [])) - set(args)
+            if missing:
+                raise ClientRefusal("missing required arguments", "ARGUMENTS_DENIED")
+            data = network.get(args["url"]) if name == "gather.fetch" else invoke(name, dict(args), root)
+            response["result"] = {"content": [{"type": "text", "text": data}], "isError": False}
+        except Exception as exc:
+            response["result"] = {"content": [{"type": "text", "text": json.dumps(
+                {"code": exc.code if isinstance(exc, ClientRefusal) else "LOCAL_PROFILE_ERROR",
+                 "detail": str(exc)})}], "isError": True}
+    else:
+        response["error"] = {"code": -32601, "message": "method not found"}
+    return response
+
+
+def install_process_boundary(network=None):
+    """Deny process and network actions in this stdlib profile, including Git."""
+    def audit(event, args):
+        if event.startswith("socket."):
+            if network is None:
+                raise PermissionError("local client profile does not grant network")
+            network.audit_socket(event, args)
+        if event in {"subprocess.Popen", "os.system", "os.posix_spawn", "os.posix_spawnp",
+                     "os.exec", "os.spawn"}:
+            raise PermissionError("local client profile does not grant processes or network")
+    sys.addaudithook(audit)
+
+
+def origin_array(value):
+    """Parse desktop setup text without interpreting it as flags or authority."""
+    if not value.strip():
+        return []
+    try:
+        values = json.loads(value)
+    except json.JSONDecodeError as exc:
+        raise argparse.ArgumentTypeError("origin setting must be a JSON array of strings") from exc
+    if not isinstance(values, list) or any(not isinstance(entry, str) for entry in values):
+        raise argparse.ArgumentTypeError("origin setting must be a JSON array of strings")
+    return values
+
+
+def main(argv=None):
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--workspace", required=True, help="explicit local directory this client may read")
+    parser.add_argument("--allow-origin", action="append", default=[], help="allow GET from this exact public HTTPS origin")
+    parser.add_argument("--allow-loopback-origin", action="append", default=[], help="allow GET from this exact literal loopback HTTP(S) origin")
+    parser.add_argument("--allow-origins-json", type=origin_array, default=[], help="desktop setup JSON array of public HTTPS origins; empty means none")
+    parser.add_argument("--allow-loopback-origins-json", type=origin_array, default=[], help="desktop setup JSON array of literal loopback origins; empty means none")
+    args = parser.parse_args(argv)
+    root_arg = Path(args.workspace).absolute()
+    root = confined(root_arg, str(root_arg))
+    if not root.is_dir():
+        parser.error("workspace must be a directory")
+    from gather.client_network import NetworkGrant
+    try:
+        network = NetworkGrant.from_launch([*args.allow_origin, *args.allow_origins_json],
+                                           [*args.allow_loopback_origin, *args.allow_loopback_origins_json])
+    except (ValueError, OSError) as exc:
+        parser.error(str(exc))
+    install_process_boundary(network)
+    # The local profile does not read permission grants from the environment.
+    for line in sys.stdin:
+        if len(line) > MAX_FILE:
+            return 2
+        try:
+            response = handle(json.loads(line), root, network)
+        except json.JSONDecodeError:
+            response = {"jsonrpc": "2.0", "id": None,
+                        "error": {"code": -32700, "message": "parse error"}}
+        if response is not None:
+            sys.stdout.write(json.dumps(response) + "\n")
+            sys.stdout.flush()
+    return 0
+
+def definitions(network=None):
+    from gather.docs_tool import annotate, docs_tool
+    tools = [annotate(docs_tool())]
+    if network is not None and network.origins:
+        tools.append(annotate({"name": "gather.fetch", "description": "Read an allowed origin; return untrusted source text and a byte receipt. No redirects or credentials.",
+                      "inputSchema": {"type": "object", "properties": {"url": {"type": "string"}}, "required": ["url"], "additionalProperties": False}}))
+    return tools
+
+
+def invoke(name, args, root):
+    from gather.docs_tool import call_docs
+    # Single files avoid a directory walk following a file link outside the grant.
+    path = confined(root, args["path"])
+    if not path.is_file():
+        raise ClientRefusal("local client intake requires one text file per call")
+    args["path"] = str(path)
+    return call_docs(args)
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

gather_engine-2.2.0/src/gather/client_network.py

@@ -0,0 +1,109 @@
+"""Launch-only, exact-origin GET grants for the read-only client profile."""
+from __future__ import annotations
+
+import ipaddress
+import json
+import socket
+import urllib.error
+import urllib.parse
+import urllib.request
+from dataclasses import dataclass
+from time import monotonic
+
+from gather.fetch import RawResponse, fetch
+from gather.grants import Grants
+from gather.net import decode_body, validate_public_http_url
+
+
+def origin_of(url):
+    if not isinstance(url, str) or any(ord(c) < 33 for c in url) or "\\" in url:
+        raise ValueError("URL must be an explicit HTTP(S) URL without whitespace")
+    parsed = urllib.parse.urlsplit(url)
+    if parsed.scheme not in {"http", "https"} or not parsed.hostname or parsed.username is not None or parsed.password is not None:
+        raise ValueError("URL credentials and non-HTTP(S) schemes are not allowed")
+    return parsed, (parsed.scheme, parsed.hostname.lower(), parsed.port or (443 if parsed.scheme == "https" else 80))
+
+
+@dataclass(frozen=True)
+class NetworkGrant:
+    origins: frozenset = frozenset()
+    addresses: frozenset = frozenset()
+
+    @classmethod
+    def from_launch(cls, public=(), loopback=()):
+        origins, addresses = set(), set()
+        for entries, local in ((public, False), (loopback, True)):
+            for url in entries:
+                parsed, origin = origin_of(url)
+                if parsed.path not in {"", "/"} or parsed.query or parsed.fragment:
+                    raise ValueError("launch grants must name an origin without a path, query or fragment")
+                if local:
+                    if not ipaddress.ip_address(origin[1]).is_loopback:
+                        raise ValueError("loopback grants require a literal loopback IP address")
+                else:
+                    if origin[0] != "https":
+                        raise ValueError("public origins require HTTPS")
+                    validate_public_http_url(url)
+                infos = socket.getaddrinfo(origin[1], origin[2], type=socket.SOCK_STREAM)
+                for info in infos:
+                    address = ipaddress.ip_address(info[4][0])
+                    if not local and not address.is_global:
+                        raise ValueError("public origin must resolve only to public addresses")
+                    addresses.add((str(address), origin[2]))
+                origins.add(origin)
+        return cls(frozenset(origins), frozenset(addresses))
+
+    def audit_socket(self, event, args):
+        if not self.origins:
+            raise PermissionError("network requires a launch grant")
+        if event == "socket.__new__":
+            if args[1] in {socket.AF_INET, socket.AF_INET6} and args[2] == socket.SOCK_STREAM:
+                return
+        elif event == "socket.getaddrinfo":
+            if any(args[0] == host and args[1] == port for _, host, port in self.origins):
+                return
+        elif event == "socket.connect":
+            destination = args[1]
+            if isinstance(destination, tuple) and (destination[0], destination[1]) in self.addresses:
+                return
+        raise PermissionError("socket operation is outside the launch grant")
+
+    def get(self, url):
+        Grants(network_sources=frozenset({"web"}) if self.origins else frozenset()).require_network("web")
+        _, origin = origin_of(url)
+        if origin not in self.origins:
+            raise ValueError("URL origin is outside the launch grant")
+
+        def transport(target, *, headers, timeout, max_bytes):
+            # No ambient proxies, credentials, cookies or redirect authority.
+            opener = urllib.request.build_opener(urllib.request.ProxyHandler({}), _NoRedirect())
+            request = urllib.request.Request(target, headers=headers)
+            try:
+                response = opener.open(request, timeout=timeout)
+            except urllib.error.HTTPError as exc:
+                exc.close()
+                raise ValueError(f"HTTP request refused or failed: {exc.code}") from None
+            with response:
+                deadline = monotonic() + 10
+                body = bytearray()
+                while True:
+                    if monotonic() >= deadline:
+                        raise ValueError("response exceeded the read time budget")
+                    chunk = response.read1(min(65536, max_bytes + 1 - len(body)))
+                    if not chunk:
+                        if response.length not in {None, 0}:
+                            raise ValueError("incomplete response; no complete receipt issued")
+                        break
+                    body.extend(chunk)
+                    if len(body) > max_bytes:
+                        raise ValueError("response exceeds client byte limit; no complete receipt issued")
+                return RawResponse(response.status, response.geturl(), tuple(response.headers.items()), bytes(body))
+
+        receipt, body = fetch(url, transport=transport, max_bytes=1_000_000, timeout=10, retries=0)
+        return json.dumps({"text": decode_body(body or b""), "receipt": receipt.as_dict(),
+                           "authority": "explicit-launch-origin", "content_trust": "untrusted-source"})
+
+
+class _NoRedirect(urllib.request.HTTPRedirectHandler):
+    def redirect_request(self, req, fp, code, msg, headers, newurl):
+        return None