phantom-scan 0.1.0__tar.gz

phantom_scan-0.1.0/PKG-INFO

@@ -0,0 +1,155 @@
+Metadata-Version: 2.4
+Name: phantom-scan
+Version: 0.1.0
+Summary: Detect divergence between a package's declared source and its published artifact
+License: MIT
+Project-URL: Source, https://github.com/pipebreach/phantom
+Project-URL: Repository, https://github.com/pipebreach/phantom
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+Provides-Extra: dev
+Requires-Dist: pytest>=8; extra == "dev"
+
+# phantom
+
+**Detects divergence between a package's declared source and the artifact it actually publishes.**
+
+Part of the [pipebreach] research project. `phantom` compares *what a package declares* (its source repository at the tag matching the published version) against *what it ships* (the wheel on PyPI). Any code present in the artifact that does not exist in the source is a **phantom** — flagged regardless of whether it "looks malicious".
+
+## Threat model
+
+An attacker compromises a package's build/publish pipeline and injects code into the distributed artifact **without touching the source repository**. Anyone auditing the code on GitHub sees nothing; anyone installing the package gets the backdoor.
+
+Real-world incidents this class of tool catches:
+
+- **LiteLLM (March 2026):** a three-stage payload (credential theft, Kubernetes lateral movement, RCE backdoor) injected into the PyPI wheel post-build. The repo was clean.
+- **XZ Utils (CVE-2024-3094):** backdoor present in the distribution tarball but not in git.
+
+Malware scanners (GuardDog, Semgrep rules, etc.) look for *known-bad patterns*. `phantom` is **pattern-agnostic**: it doesn't care whether injected code looks bad — only that it isn't in the source. That catches clean, obfuscated, or never-before-seen injection. Reproducible-builds infrastructure (rebuilderd) solves this for OS distros; nothing equivalent exists for language ecosystems. That's the gap.
+
+## Quickstart
+
+Requires Python 3.11+. Zero runtime dependencies (stdlib only, by design).
+
+```bash
+pip install -e .
+
+# Scan a single package version
+phantom scan requests==2.31.0
+
+# Machine-readable output
+phantom scan requests==2.31.0 --json
+phantom scan requests==2.31.0 --sarif   # for GitHub code scanning
+```
+
+What a scan does:
+
+1. Downloads the pure-Python wheel from the PyPI JSON API.
+2. Resolves the source repo from the package metadata (`project_urls`) and finds the tag matching the version (`v{ver}`, `{ver}`, `release-{ver}`).
+3. Computes an **AST-normalized hash** of every `.py` file on both sides (comments/whitespace/formatting don't count) and flags every wheel file whose content exists nowhere in the source.
+4. Grades each phantom by its execution vectors (subprocess, network, `exec`/`eval`, `os.environ` access). Reading data + network egress = `critical` (exfiltration shape).
+5. Any `.pth` file inside a wheel is always flagged (it executes at interpreter startup).
+
+Downloads are cached on disk (`~/.cache/phantom` by default, `--cache-dir` to override), so re-scanning the same `pkg==version` is deterministic and offline.
+
+## Exit codes
+
+| Code | Meaning |
+|------|---------|
+| 0 | No findings (or only medium/low severity) |
+| 1 | At least one **high**/**critical** finding |
+| 2 | Execution error (bad arguments, network failure, package not found) |
+| 3 | Out of scope: the release has no pure-Python wheel (binary wheels or sdist only) |
+
+## GitHub Action
+
+Package maintainers can verify their own releases right after publishing —
+this catches a compromised build/publish pipeline (the LiteLLM case) even
+though the repo looks clean:
+
+```yaml
+name: verify-release
+on:
+  workflow_run:
+    workflows: [publish]        # run after your publish workflow completes
+    types: [completed]
+
+jobs:
+  phantom:
+    runs-on: ubuntu-latest
+    permissions:
+      security-events: write    # for SARIF upload
+    steps:
+      - uses: pipebreach/phantom@main # pin to a release commit SHA
+        id: scan
+        with:
+          spec: mypkg==1.2.3    # e.g. derived from the release tag
+      - uses: github/codeql-action/upload-sarif@411c4c9a36b3fca4d674f06b6396b2c6d23522c6 # v3.36.3
+        if: always() && hashFiles('phantom-results.sarif') != ''
+        with:
+          sarif_file: phantom-results.sarif
+```
+
+The job fails when phantom finds a high/critical divergence. Notes:
+
+- Scan *after* the artifact is live on PyPI (publishing and scanning in the
+  same workflow can race index propagation).
+- If your tags carry a `v` prefix, strip it for the spec: phantom probes tag
+  conventions on the source side, but the spec version must match PyPI.
+- Releases without a pure-Python wheel emit a warning instead of failing;
+  set `fail-on-out-of-scope: "true"` to make them fail.
+
+Inputs: `spec` (required), `ecosystem` (default `pypi`), `sarif-file`
+(default `phantom-results.sarif`), `fail-on-out-of-scope` (default `false`).
+Outputs: `exit-code`, `sarif-file`.
+
+Scanning a whole lockfile from a consumer project (`phantom audit`) will use
+the same action once M2 lands.
+
+## Finding types
+
+| Type | Meaning | Severity |
+|------|---------|----------|
+| `phantom_file` | A `.py` in the wheel whose content exists nowhere in the source | `medium`–`critical` per execution vectors |
+| `suspicious_pth` | A `.pth` file shipped inside the wheel | `high`; `critical` if it has import lines |
+| `no_source_declared` | No source repo in the package metadata — unverifiable by construction | `high` |
+| `source_ref_not_found` | Repo declared, but no tag matches the version | `medium` |
+
+Every finding carries a `confidence` and a `reason`: phantom prefers saying "this is phantom but might be build-generated code" (e.g. `_version.py` from setuptools-scm gets `low` confidence) over false certainty.
+
+The `--json` output is a versioned public contract (`schema_version`); breaking changes bump the version.
+
+## Library use
+
+The CLI is a thin layer over the core (usable in CI, batch jobs, services):
+
+```python
+from phantom.cache import DiskCache, default_cache_dir
+from phantom.core import scan
+from phantom.registry import build_default_registry
+
+registry = build_default_registry(DiskCache(default_cache_dir()))
+result = scan("six", "1.16.0", registry.get("pypi"))
+print(result.to_dict())
+```
+
+## Known limits (M1)
+
+Explicitly **not supported yet** — the plugin architecture (`Ecosystem`/`Fetcher`/`SourceResolver`/`Normalizer` interfaces) is designed so these land without touching the core:
+
+- **Compiled/binary wheels and sdist-only releases** — reported as out of scope (exit 3). Needs build-step normalizers (M3).
+- **npm / JavaScript** — no minification/transpilation normalizers yet (M2/M3).
+- **Phantom *spans*** — code injected *into* a file that does exist in the source. M1 only detects whole phantom files; intra-file AST diffing is M2. A modified file will still show up as a phantom (its hash won't match), but without span-level localization.
+- **`phantom audit <lockfile>`** — registered but stubbed (M2).
+- **Non-GitHub forges** (GitLab, Codeberg…) — M3.
+- **Tag-based resolution only** — packages that publish from a commit without tagging that version yield `source_ref_not_found`.
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+pytest                             # unit tests, fully offline
+PHANTOM_NETWORK_TESTS=1 pytest -m network   # integration tests against real PyPI/GitHub
+```
+
+[pipebreach]: https://github.com/pipebreach

phantom_scan-0.1.0/README.md

@@ -0,0 +1,143 @@
+# phantom
+
+**Detects divergence between a package's declared source and the artifact it actually publishes.**
+
+Part of the [pipebreach] research project. `phantom` compares *what a package declares* (its source repository at the tag matching the published version) against *what it ships* (the wheel on PyPI). Any code present in the artifact that does not exist in the source is a **phantom** — flagged regardless of whether it "looks malicious".
+
+## Threat model
+
+An attacker compromises a package's build/publish pipeline and injects code into the distributed artifact **without touching the source repository**. Anyone auditing the code on GitHub sees nothing; anyone installing the package gets the backdoor.
+
+Real-world incidents this class of tool catches:
+
+- **LiteLLM (March 2026):** a three-stage payload (credential theft, Kubernetes lateral movement, RCE backdoor) injected into the PyPI wheel post-build. The repo was clean.
+- **XZ Utils (CVE-2024-3094):** backdoor present in the distribution tarball but not in git.
+
+Malware scanners (GuardDog, Semgrep rules, etc.) look for *known-bad patterns*. `phantom` is **pattern-agnostic**: it doesn't care whether injected code looks bad — only that it isn't in the source. That catches clean, obfuscated, or never-before-seen injection. Reproducible-builds infrastructure (rebuilderd) solves this for OS distros; nothing equivalent exists for language ecosystems. That's the gap.
+
+## Quickstart
+
+Requires Python 3.11+. Zero runtime dependencies (stdlib only, by design).
+
+```bash
+pip install -e .
+
+# Scan a single package version
+phantom scan requests==2.31.0
+
+# Machine-readable output
+phantom scan requests==2.31.0 --json
+phantom scan requests==2.31.0 --sarif   # for GitHub code scanning
+```
+
+What a scan does:
+
+1. Downloads the pure-Python wheel from the PyPI JSON API.
+2. Resolves the source repo from the package metadata (`project_urls`) and finds the tag matching the version (`v{ver}`, `{ver}`, `release-{ver}`).
+3. Computes an **AST-normalized hash** of every `.py` file on both sides (comments/whitespace/formatting don't count) and flags every wheel file whose content exists nowhere in the source.
+4. Grades each phantom by its execution vectors (subprocess, network, `exec`/`eval`, `os.environ` access). Reading data + network egress = `critical` (exfiltration shape).
+5. Any `.pth` file inside a wheel is always flagged (it executes at interpreter startup).
+
+Downloads are cached on disk (`~/.cache/phantom` by default, `--cache-dir` to override), so re-scanning the same `pkg==version` is deterministic and offline.
+
+## Exit codes
+
+| Code | Meaning |
+|------|---------|
+| 0 | No findings (or only medium/low severity) |
+| 1 | At least one **high**/**critical** finding |
+| 2 | Execution error (bad arguments, network failure, package not found) |
+| 3 | Out of scope: the release has no pure-Python wheel (binary wheels or sdist only) |
+
+## GitHub Action
+
+Package maintainers can verify their own releases right after publishing —
+this catches a compromised build/publish pipeline (the LiteLLM case) even
+though the repo looks clean:
+
+```yaml
+name: verify-release
+on:
+  workflow_run:
+    workflows: [publish]        # run after your publish workflow completes
+    types: [completed]
+
+jobs:
+  phantom:
+    runs-on: ubuntu-latest
+    permissions:
+      security-events: write    # for SARIF upload
+    steps:
+      - uses: pipebreach/phantom@main # pin to a release commit SHA
+        id: scan
+        with:
+          spec: mypkg==1.2.3    # e.g. derived from the release tag
+      - uses: github/codeql-action/upload-sarif@411c4c9a36b3fca4d674f06b6396b2c6d23522c6 # v3.36.3
+        if: always() && hashFiles('phantom-results.sarif') != ''
+        with:
+          sarif_file: phantom-results.sarif
+```
+
+The job fails when phantom finds a high/critical divergence. Notes:
+
+- Scan *after* the artifact is live on PyPI (publishing and scanning in the
+  same workflow can race index propagation).
+- If your tags carry a `v` prefix, strip it for the spec: phantom probes tag
+  conventions on the source side, but the spec version must match PyPI.
+- Releases without a pure-Python wheel emit a warning instead of failing;
+  set `fail-on-out-of-scope: "true"` to make them fail.
+
+Inputs: `spec` (required), `ecosystem` (default `pypi`), `sarif-file`
+(default `phantom-results.sarif`), `fail-on-out-of-scope` (default `false`).
+Outputs: `exit-code`, `sarif-file`.
+
+Scanning a whole lockfile from a consumer project (`phantom audit`) will use
+the same action once M2 lands.
+
+## Finding types
+
+| Type | Meaning | Severity |
+|------|---------|----------|
+| `phantom_file` | A `.py` in the wheel whose content exists nowhere in the source | `medium`–`critical` per execution vectors |
+| `suspicious_pth` | A `.pth` file shipped inside the wheel | `high`; `critical` if it has import lines |
+| `no_source_declared` | No source repo in the package metadata — unverifiable by construction | `high` |
+| `source_ref_not_found` | Repo declared, but no tag matches the version | `medium` |
+
+Every finding carries a `confidence` and a `reason`: phantom prefers saying "this is phantom but might be build-generated code" (e.g. `_version.py` from setuptools-scm gets `low` confidence) over false certainty.
+
+The `--json` output is a versioned public contract (`schema_version`); breaking changes bump the version.
+
+## Library use
+
+The CLI is a thin layer over the core (usable in CI, batch jobs, services):
+
+```python
+from phantom.cache import DiskCache, default_cache_dir
+from phantom.core import scan
+from phantom.registry import build_default_registry
+
+registry = build_default_registry(DiskCache(default_cache_dir()))
+result = scan("six", "1.16.0", registry.get("pypi"))
+print(result.to_dict())
+```
+
+## Known limits (M1)
+
+Explicitly **not supported yet** — the plugin architecture (`Ecosystem`/`Fetcher`/`SourceResolver`/`Normalizer` interfaces) is designed so these land without touching the core:
+
+- **Compiled/binary wheels and sdist-only releases** — reported as out of scope (exit 3). Needs build-step normalizers (M3).
+- **npm / JavaScript** — no minification/transpilation normalizers yet (M2/M3).
+- **Phantom *spans*** — code injected *into* a file that does exist in the source. M1 only detects whole phantom files; intra-file AST diffing is M2. A modified file will still show up as a phantom (its hash won't match), but without span-level localization.
+- **`phantom audit <lockfile>`** — registered but stubbed (M2).
+- **Non-GitHub forges** (GitLab, Codeberg…) — M3.
+- **Tag-based resolution only** — packages that publish from a commit without tagging that version yield `source_ref_not_found`.
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+pytest                             # unit tests, fully offline
+PHANTOM_NETWORK_TESTS=1 pytest -m network   # integration tests against real PyPI/GitHub
+```
+
+[pipebreach]: https://github.com/pipebreach

phantom_scan-0.1.0/phantom/__init__.py

@@ -0,0 +1,12 @@
+"""phantom — detect divergence between a package's source and its published artifact.
+
+Core entry point for library use::
+
+    from phantom.core import scan
+    from phantom.registry import build_default_registry
+
+    registry = build_default_registry()
+    result = scan("some-pkg", "1.2.3", registry.get("pypi"))
+"""
+
+__version__ = "0.1.0"

phantom_scan-0.1.0/phantom/cache.py

@@ -0,0 +1,63 @@
+"""Disk cache keyed by URL digest and the single HTTP helper. A cached
+scan of the same ``pkg==version`` is deterministic and works offline."""
+
+from __future__ import annotations
+
+import hashlib
+import urllib.error
+import urllib.request
+from pathlib import Path
+
+from phantom.errors import FetchError, NotFoundError
+
+USER_AGENT = "phantom/0.1 (+https://github.com/pipebreach/phantom)"
+
+
+def default_cache_dir() -> Path:
+    return Path.home() / ".cache" / "phantom"
+
+
+class DiskCache:
+    """Blob store keyed by the SHA-256 of an arbitrary string key."""
+
+    def __init__(self, root: Path) -> None:
+        self.root = root
+
+    def _path(self, key: str) -> Path:
+        digest = hashlib.sha256(key.encode("utf-8")).hexdigest()
+        return self.root / digest[:2] / digest
+
+    def get(self, key: str) -> bytes | None:
+        path = self._path(key)
+        if path.is_file():
+            return path.read_bytes()
+        return None
+
+    def put(self, key: str, data: bytes) -> None:
+        path = self._path(key)
+        path.parent.mkdir(parents=True, exist_ok=True)
+        tmp = path.with_suffix(".tmp")
+        tmp.write_bytes(data)
+        tmp.replace(path)
+
+
+def http_get(url: str, cache: DiskCache | None = None, timeout: float = 60.0) -> bytes:
+    """GET a URL through the cache. Raises ``NotFoundError`` on 404,
+    ``FetchError`` on any other failure."""
+    if cache is not None:
+        cached = cache.get(url)
+        if cached is not None:
+            return cached
+    request = urllib.request.Request(url, headers={"User-Agent": USER_AGENT})
+    try:
+        with urllib.request.urlopen(request, timeout=timeout) as response:
+            data = response.read()
+    except urllib.error.HTTPError as exc:
+        if exc.code == 404:
+            raise NotFoundError(f"404 Not Found: {url}") from exc
+        raise FetchError(f"HTTP {exc.code} fetching {url}") from exc
+    except urllib.error.URLError as exc:
+        raise FetchError(f"network error fetching {url}: {exc.reason}") from exc
+    if cache is not None:
+        cache.put(url, data)
+    return data

phantom_scan-0.1.0/phantom/cli.py

@@ -0,0 +1,133 @@
+"""Thin CLI layer over ``phantom.core.scan``. Exit codes: 0 clean, 1 findings
+of high/critical severity, 2 execution error, 3 out of scope."""
+
+from __future__ import annotations
+
+import argparse
+import sys
+from pathlib import Path
+
+from phantom import __version__, core
+from phantom.cache import DiskCache, default_cache_dir
+from phantom.errors import OutOfScopeError, PhantomError
+from phantom.registry import Registry, build_default_registry
+from phantom.report import json_report, sarif_report, table_report
+
+EXIT_OK = 0
+EXIT_FINDINGS = 1
+EXIT_ERROR = 2
+EXIT_OUT_OF_SCOPE = 3
+
+_EXIT_CODES_HELP = """\
+exit codes:
+  0  no findings (or only medium/low severity)
+  1  at least one high/critical finding
+  2  execution error
+  3  package out of scope (no pure-Python wheel available)
+"""
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(
+        prog="phantom",
+        description=(
+            "Detect divergence between a package's declared source and its "
+            "published artifact."
+        ),
+        epilog=_EXIT_CODES_HELP,
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+    )
+    parser.add_argument("--version", action="version", version=f"phantom {__version__}")
+    subparsers = parser.add_subparsers(dest="command", required=True)
+
+    scan = subparsers.add_parser(
+        "scan",
+        help="scan a single package version (e.g. phantom scan requests==2.31.0)",
+        epilog=_EXIT_CODES_HELP,
+        formatter_class=argparse.RawDescriptionHelpFormatter,
+    )
+    scan.add_argument("spec", help="package spec in the form <pkg>==<version>")
+    scan.add_argument(
+        "--ecosystem",
+        default="pypi",
+        help="package ecosystem (default: pypi)",
+    )
+    output = scan.add_mutually_exclusive_group()
+    output.add_argument(
+        "--json", action="store_true", help="emit the versioned JSON schema"
+    )
+    output.add_argument(
+        "--sarif", action="store_true", help="emit SARIF 2.1.0 for code scanning"
+    )
+    scan.add_argument(
+        "--cache-dir",
+        type=Path,
+        default=None,
+        help=f"download cache directory (default: {default_cache_dir()})",
+    )
+
+    audit = subparsers.add_parser(
+        "audit",
+        help="scan every package in a lockfile (requirements.txt / poetry.lock) [M2]",
+    )
+    audit.add_argument("lockfile", type=Path, help="path to the lockfile")
+
+    return parser
+
+
+def run(argv: list[str] | None = None, registry: Registry | None = None) -> int:
+    """Parse arguments and execute; ``registry`` is injectable for tests."""
+    args = build_parser().parse_args(argv)
+
+    if args.command == "audit":
+        print(
+            "phantom audit is not implemented yet (planned for M2).",
+            file=sys.stderr,
+        )
+        return EXIT_ERROR
+
+    try:
+        package, version = _parse_spec(args.spec)
+    except ValueError as exc:
+        print(f"error: {exc}", file=sys.stderr)
+        return EXIT_ERROR
+
+    if registry is None:
+        cache = DiskCache(args.cache_dir or default_cache_dir())
+        registry = build_default_registry(cache)
+
+    try:
+        ecosystem = registry.get(args.ecosystem)
+        result = core.scan(package, version, ecosystem)
+    except OutOfScopeError as exc:
+        print(f"out of scope: {exc}", file=sys.stderr)
+        return EXIT_OUT_OF_SCOPE
+    except PhantomError as exc:
+        print(f"error: {exc}", file=sys.stderr)
+        return EXIT_ERROR
+
+    if args.json:
+        print(json_report.render(result))
+    elif args.sarif:
+        print(sarif_report.render(result))
+    else:
+        print(table_report.render(result))
+
+    return core.exit_code_for(result)
+
+
+def _parse_spec(spec: str) -> tuple[str, str]:
+    package, sep, version = spec.partition("==")
+    if not sep or not package or not version:
+        raise ValueError(
+            f"invalid spec {spec!r}: expected exact pin <pkg>==<version>"
+        )
+    return package.strip(), version.strip()
+
+
+def main() -> None:
+    sys.exit(run())
+
+
+if __name__ == "__main__":
+    main()

phantom_scan-0.1.0/phantom/core.py

@@ -0,0 +1,81 @@
+"""Orchestrator and library entry point: drives the ``Ecosystem`` interfaces
+only, holds no state."""
+
+from __future__ import annotations
+
+from phantom import differ
+from phantom.ecosystems.base import Ecosystem
+from phantom.models import (
+    Confidence,
+    Finding,
+    FindingType,
+    NoSource,
+    ScanResult,
+    Severity,
+    SourceStatus,
+)
+
+
+def scan(package: str, version: str, ecosystem: Ecosystem) -> ScanResult:
+    """Scan one package version for source/artifact divergence.
+
+    Propagates fetcher errors; unresolvable source is a finding, not an error.
+    """
+    artifact = ecosystem.fetcher.fetch_artifact(package, version)
+    source = ecosystem.source_resolver.resolve_source(
+        package, version, artifact.metadata
+    )
+
+    if isinstance(source, NoSource):
+        return _no_source_result(package, version, ecosystem.name, source)
+
+    outcome = differ.diff(artifact, source, ecosystem.normalizers)
+    return ScanResult(
+        package=package,
+        version=version,
+        ecosystem=ecosystem.name,
+        source_status=SourceStatus.RESOLVED,
+        source_repo=source.repo_url,
+        source_ref=source.ref,
+        findings=outcome.findings,
+        files_scanned=outcome.files_scanned,
+    )
+
+
+def exit_code_for(result: ScanResult) -> int:
+    """Return 1 if any finding is high or critical, else 0."""
+    if any(f.severity in (Severity.HIGH, Severity.CRITICAL) for f in result.findings):
+        return 1
+    return 0
+
+
+def _no_source_result(
+    package: str, version: str, ecosystem: str, source: NoSource
+) -> ScanResult:
+    if source.finding_type == FindingType.NO_SOURCE_DECLARED:
+        # Unverifiable by construction: a risk in itself.
+        status = SourceStatus.NO_SOURCE_DECLARED
+        severity = Severity.HIGH
+        confidence = Confidence.HIGH
+    else:
+        # A repo exists but no tag convention matched; flag softly.
+        status = SourceStatus.REF_NOT_FOUND
+        severity = Severity.MEDIUM
+        confidence = Confidence.MEDIUM
+    finding = Finding(
+        type=source.finding_type,
+        path=None,
+        severity=severity,
+        confidence=confidence,
+        reason=source.detail,
+    )
+    return ScanResult(
+        package=package,
+        version=version,
+        ecosystem=ecosystem,
+        source_status=status,
+        source_repo=source.repo_url,
+        source_ref=None,
+        findings=[finding],
+        files_scanned=0,
+    )