ghostpkg 0.19.2__tar.gz → 0.21.0__tar.gz

{ghostpkg-0.19.2 → ghostpkg-0.21.0}/.gitignore

@@ -67,3 +67,6 @@ PLAN.md
 
 # Banner art-direction options (working files, not shipped)
 banner-options/
+
+# Agent scratch worktrees
+.claude/

{ghostpkg-0.19.2 → ghostpkg-0.21.0}/CHANGELOG.md

@@ -6,6 +6,122 @@ this project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
 
+## [0.21.0] - 2026-09-04
+
+Three more false blocks, all found by a new release gate that scans large real
+repositories instead of synthetic manifests. On `home-assistant/core` alone the
+count went **39 -> 0**.
+
+### Added
+- **`scripts/fieldtest.py`, a release gate.** It clones five repositories chosen
+  for the dependency shapes they contain, scans each, and fails on any block
+  not listed with a reason. Every name in those projects is something thousands
+  of people install daily, so **every block is false until shown otherwise**.
+  Current state: **4,461 packages checked, zero blocks.**
+
+  This exists because 605 unit tests and a 35-check acceptance pass had missed
+  eleven defects that one pass over real repositories found immediately.
+
+### Fixed
+- **A constraints file forbids; it does not install.** `pip` never installs
+  from one -- it only bounds a version if the package arrives some other way --
+  so the standard way to forbid a package outright is to pin it to a version
+  that cannot exist. Home Assistant does this for eight of them
+  (`pycrypto==1000000000.0.0`), and checking those pins turned deliberate
+  exclusions into **35 reported blocks**. The flag now survives nested `-c` and
+  `-r` includes. The name in a constraints file is still checked.
+- **PEP 440 pads the release segment with zeros.** `0.8` and `0.8.0` are one
+  version, as are `1.6.6` and `1.6.6.0`. Both spellings sit in Home Assistant
+  requirements against packages that store the other form, and comparing the
+  raw text blocked `libsoundtouch` and `baidu-aip`. Worse, the test suite had
+  asserted the *opposite* -- that `1.0` and `1.0.0` differ -- so the wrong
+  behaviour was locked in by a test. Corrected, and the padding is now
+  asserted.
+- **A block no longer rests on a cached version list.** An established package
+  is cached for a day, so a release published inside that window is invisible;
+  `opower==0.21.0` exists and was blocked from a stale list. The version list
+  is now re-fetched before any version block, which is the same rule that
+  already governs negative answers: the answer that blocks has to be fresh.
+
+614 tests.
+
+## [0.20.0] - 2026-09-03
+
+Four defects found by scanning **10,109 packages** across `vercel/next.js`,
+`home-assistant/core` and `getsentry/sentry`. Those three repositories produced
+53 blocks and **every one of them was false**. None of these shapes occur in a
+small synthetic manifest, which is why 571 tests and a 35-check acceptance pass
+had missed all four.
+
+### Fixed
+- **`ghostpkg check -e npm "@scope/name"` reported success having checked
+  nothing.** The names went through the PEP 508 requirements parser, whose
+  pattern demands an alphanumeric first character, so every scoped npm name was
+  dropped in silence -- the run printed "all 0 packages look fine" and exited
+  0. On the command this tool is named for, and for a quarter of the npm
+  namespace. A false all-clear is worse than any false positive, and this is
+  the most serious defect the project has shipped.
+- **A pinned version was compared as text rather than as a version.**
+  `aiopurpleair==2025.08.1` was blocked as non-existent while `pip download`
+  installed it happily: PyPI stores the canonical `2025.8.1`. Comparison now
+  normalises both sides -- leading zeros, a leading `v`, `-`/`_` separators and
+  the pre-release spellings. Three such pins sit in unmodified Home Assistant
+  requirements, and a false block breaks a build.
+- **A package the checkout provides itself is no longer looked up.** All three
+  names blocked in a 6,335-package scan of `vercel/next.js` were the
+  repository's own packages, `@next/font` among them. A monorepo depends on
+  itself, and not always through `workspace:*` -- an exact pin is just as
+  common. Names declared by any `package.json` or `pyproject.toml` in the scan
+  are now excluded, which is the same rule as `workspace:` stated differently.
+- **`MANIFEST.in` is no longer read as a requirements file.** `.in` is the
+  pip-tools convention and also the extension of a packaging directives file.
+  Read as requirements it reported `graft` as a package that exists -- there is
+  a real project of that name -- and blocked `recursive-exclude`.
+
+### Documentation
+- The `--json` example showed the bare array from before 0.14.0. It is a
+  `{schema, tool, summary, findings}` envelope, and the documented example now
+  matches, including `source` and `line`.
+
+605 tests.
+
+## [0.19.3] - 2026-09-03
+
+Found by reviewing the published 0.19.2 rather than the working tree. Two of
+the four are false blocks, which this project treats as its worst failure, and
+both have the same cause: `jslocks.py` was written after the test file that
+exists to defend the rule "a dependency naming its own source is not the
+registry's business", so `yarn.lock` and `pnpm-lock.yaml` were never covered by
+it.
+
+### Fixed
+- **A yarn `owner/repo#ref` dependency was blocked.** GitHub shorthand was
+  handled for `package.json` and missed in `yarn.lock`, so
+  `internal-lib@acme/internal-lib#v1.2.3` was looked up on npmjs, found absent,
+  and reported as a package that does not exist.
+- **A pnpm git or URL dependency produced a nonsense package name.** The
+  protocol is only a prefix in lockfile v5; from v6 it follows `name@`, so the
+  guard never fired. `github.com/acme/forked/abc123` was read as a package
+  called `github.com/acme/forked`, and
+  `foo@https://codeload.github.com/...` as one called
+  `foo@https://codeload.github.com/acme/foo`. Both were then blocked.
+- **`--timeout` did nothing.** `def _get_json(url, timeout=TIMEOUT)` bound the
+  module global once, at import; the CLI assigned `registries.TIMEOUT` and
+  nothing ever re-read it, so every request used 15 seconds regardless.
+- **`.windsurfrules` was found and never scanned.** The directory search
+  offered it up, the parser refused it, and the CLI ignores an unreadable
+  *discovered* file on purpose -- so nothing was printed. There is now a test
+  asserting that every file the search returns can actually be parsed.
+
+### Added
+- End-to-end tests for suppression. The ignore file was loaded and applied in
+  `main()` only, and nothing asserted its effect on a verdict or an exit code:
+  the entire policy call could be replaced with `used = []` and all 539 tests
+  still passed. Three tests now cover the real path, and they fail against that
+  stub.
+
+571 tests.
+
 ## [0.19.2] - 2026-09-02
 
 Found by throwing hostile input at the parsers rather than by reading them.
@@ -733,7 +849,10 @@ First release.
 - No corpus of hallucinated package names is shipped, following the decision of
   the USENIX'25 authors not to publish theirs.
 
-[Unreleased]: https://github.com/M1rwana12/ghostpkg/compare/v0.19.2...HEAD
+[Unreleased]: https://github.com/M1rwana12/ghostpkg/compare/v0.21.0...HEAD
+[0.21.0]: https://github.com/M1rwana12/ghostpkg/releases/tag/v0.21.0
+[0.20.0]: https://github.com/M1rwana12/ghostpkg/releases/tag/v0.20.0
+[0.19.3]: https://github.com/M1rwana12/ghostpkg/releases/tag/v0.19.3
 [0.19.2]: https://github.com/M1rwana12/ghostpkg/releases/tag/v0.19.2
 [0.19.1]: https://github.com/M1rwana12/ghostpkg/releases/tag/v0.19.1
 [0.19.0]: https://github.com/M1rwana12/ghostpkg/releases/tag/v0.19.0

{ghostpkg-0.19.2 → ghostpkg-0.21.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: ghostpkg
-Version: 0.19.2
+Version: 0.21.0
 Summary: Catch package names that do not exist before you install them
 Project-URL: Homepage, https://github.com/m1rwana12/ghostpkg
 Project-URL: Issues, https://github.com/m1rwana12/ghostpkg/issues
@@ -340,19 +340,32 @@ malformed file **stops the run** rather than quietly leaving you unprotected.
 ### JSON output
 
 ```console
-$ ghostpkg check somepkgthatisnotreal9911 --json
-[
-  {
-    "name": "somepkgthatisnotreal9911",
-    "ecosystem": "pypi",
-    "verdict": "BLOCK",
-    "reasons": [
-      "does not exist on pypi"
-    ]
-  }
-]
+$ ghostpkg scan requirements.txt --json
+{
+  "schema": 1,
+  "tool": { "name": "ghostpkg", "version": "0.20.0" },
+  "summary": { "checked": 2, "blocked": 1, "warned": 0, "errored": 0 },
+  "findings": [
+    {
+      "name": "somepkgthatisnotreal9911",
+      "ecosystem": "pypi",
+      "verdict": "BLOCK",
+      "source": "requirements.txt",
+      "line": 2,
+      "exists": false,
+      "latest_version": null,
+      "reasons": [
+        { "rule": "GP001", "text": "does not exist on pypi" }
+      ]
+    }
+  ]
+}
 ```
 
+`schema` is there so a consumer can tell what it is reading; `source` and
+`line` say where the name was written. `--format github` emits workflow
+commands instead, and `--format text` is the default.
+
 ---
 
 ## Every signal it checks

{ghostpkg-0.19.2 → ghostpkg-0.21.0}/README.en.md

@@ -292,19 +292,32 @@ malformed file **stops the run** rather than quietly leaving you unprotected.
 ### JSON output
 
 ```console
-$ ghostpkg check somepkgthatisnotreal9911 --json
-[
-  {
-    "name": "somepkgthatisnotreal9911",
-    "ecosystem": "pypi",
-    "verdict": "BLOCK",
-    "reasons": [
-      "does not exist on pypi"
-    ]
-  }
-]
+$ ghostpkg scan requirements.txt --json
+{
+  "schema": 1,
+  "tool": { "name": "ghostpkg", "version": "0.20.0" },
+  "summary": { "checked": 2, "blocked": 1, "warned": 0, "errored": 0 },
+  "findings": [
+    {
+      "name": "somepkgthatisnotreal9911",
+      "ecosystem": "pypi",
+      "verdict": "BLOCK",
+      "source": "requirements.txt",
+      "line": 2,
+      "exists": false,
+      "latest_version": null,
+      "reasons": [
+        { "rule": "GP001", "text": "does not exist on pypi" }
+      ]
+    }
+  ]
+}
 ```
 
+`schema` is there so a consumer can tell what it is reading; `source` and
+`line` say where the name was written. `--format github` emits workflow
+commands instead, and `--format text` is the default.
+
 ---
 
 ## Every signal it checks

{ghostpkg-0.19.2 → ghostpkg-0.21.0}/ghostpkg/__init__.py

@@ -1,6 +1,6 @@
 """ghostpkg -- catch package names that do not exist before you install them."""
 
-__version__ = "0.19.2"
+__version__ = "0.21.0"
 
 from .assess import Finding, Verdict, assess
 from .registries import PackageFacts, RegistryError, fetch

{ghostpkg-0.19.2 → ghostpkg-0.21.0}/ghostpkg/cli.py

@@ -15,7 +15,13 @@ from . import __version__, registries
 from .assess import Finding
 from .cache import Cache
 from .discover import discover
-from .manifests import UnsupportedManifest, load_manifest, parse_requirements
+from .manifests import (
+    UnsupportedManifest,
+    declared_name,
+    load_manifest,
+    parse_npm_names,
+    parse_requirements,
+)
 from .policy import PolicyError, apply as apply_policy, load as load_policy
 from .report import Palette, as_github, as_json, render, summarise, use_colour
 from .scanner import DEFAULT_WORKERS, evaluate
@@ -134,6 +140,16 @@ def main(argv: list[str] | None = None) -> int:
             print(f"ghostpkg: no manifests found in {where}", file=sys.stderr)
             return EXIT_NOTHING_SCANNED
 
+        # Names the checkout provides itself. A monorepo depends on its own
+            # packages, and not always through `workspace:*` -- an exact pin is
+            # just as common, and looking those up on the public registry
+            # blocked every one of them.
+        local: set[str] = set()
+        for path in paths:
+            own = declared_name(path)
+            if own:
+                local.add(own.lower())
+
         for path in paths:
             try:
                 found, ecosystem = load_manifest(path)
@@ -151,6 +167,8 @@ def main(argv: list[str] | None = None) -> int:
                     return EXIT_ERROR
                 continue
             for requirement in found:
+                if requirement.name.lower() in local:
+                    continue  # provided by this checkout
                 by_ecosystem.setdefault(
                     requirement.ecosystem or ecosystem, []
                 ).append(requirement)
@@ -162,7 +180,15 @@ def main(argv: list[str] | None = None) -> int:
             return EXIT_NOTHING_SCANNED
     else:
         # Accept a pin on the command line too: `ghostpkg check requests==2.31.0`.
-        by_ecosystem[args.ecosystem] = parse_requirements("\n".join(args.names))
+        # npm names go through their own reader, because a scoped name is not
+        # expressible in PEP 508 and the requirements parser dropped every one.
+        if args.ecosystem == "npm":
+            by_ecosystem[args.ecosystem] = parse_npm_names(args.names)
+        else:
+            by_ecosystem[args.ecosystem] = parse_requirements("\n".join(args.names))
+        if not by_ecosystem[args.ecosystem]:
+            print(f"ghostpkg: no package names in {' '.join(args.names)}", file=sys.stderr)
+            return EXIT_NOTHING_SCANNED
 
     try:
         policy, policy_path = load_policy(args.config)