ghostpkg 0.20.0__tar.gz → 0.22.0__tar.gz

{ghostpkg-0.20.0 → ghostpkg-0.22.0}/.gitignore

@@ -67,3 +67,6 @@ PLAN.md
 
 # Banner art-direction options (working files, not shipped)
 banner-options/
+
+# Agent scratch worktrees
+.claude/

{ghostpkg-0.20.0 → ghostpkg-0.22.0}/CHANGELOG.md

@@ -6,6 +6,98 @@ this project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [Unreleased]
 
+## [0.22.0] - 2026-09-04
+
+Four agents scanned **21 repositories and 78,000 package names**. They found
+**51 false blocks** and not one true positive among them, plus twelve places
+where the documentation described a different tool.
+
+### Fixed -- false blocks
+- **pnpm v5 keys with a peer suffix (17 blocks).** `/react-dom/18.2.0_react@18.2.0`
+  was read as a package called `react-dom/18.2.0_react`. The peer suffix carries
+  an `@digit` of its own, so hunting for the first one found the wrong `@`. The
+  version is now located by structure -- in a v5 key the last slash-separated
+  segment starts with a digit -- which also fixed `eslint-plugin-react`,
+  `tsutils` and `@babel/helper-compilation-targets`.
+- **pnpm `name@file:path` keys (8 blocks).** The protocol is only a prefix in
+  v5; from v6 it follows `name@`. A protocol is now looked for anywhere in the
+  key, since a `:` cannot occur in a published npm name.
+- **A yarn `npm:v1.1.0` range (2 blocks).** The leading `v` made a version range
+  look like an alias to a package called `v1.1.0`.
+- **Checksum files read as requirements (5 blocks).** Airflow keeps 127 files
+  whose entire content is one MD5; five carry `constraints` in the name, and a
+  32-character hex string is a legal PEP 508 name.
+- **Materialised symlinks (4 blocks).** Git writes a symlink as a plain file
+  holding its target where the filesystem has no symlink support, so
+  `requirements_compiled_py3.10.txt` contained the line
+  `requirements_compiled.txt` -- read as a package.
+- **PEP 440 local versions (17 blocks).** PyPI refuses an upload carrying one,
+  so `torch==2.9.0+cu128` can never match a release list. Ray pins every CUDA
+  build that way.
+
+The first attempt at the last two rejected any name ending in `.yaml`, `.cfg`
+or `.ini` as well. `ruamel.yaml`, `ruamel.yaml.clib` and `pytest.ini` are all
+real packages, so that would have traded a false block for a silent miss on
+three of them. The guard is now two extensions and only on a line with no
+version.
+
+### Fixed -- flags
+- **`--timeout` now reaches `--deep`**, which downloaded archives on a fixed
+  budget of its own, and **`--timeout 0` is refused** rather than silently
+  ignored.
+
+### Documentation
+Both READMEs stated that **"does not exist" is cached for an hour** and then, 300
+lines later, that negatives are never cached. The second is true; the first
+described the precise failure this tool exists to avoid. Also corrected: `-e`
+does not apply to `scan`; `--format` and `--config` were in no options table;
+"only one release" and "no repository link" are age-gated; the action's
+`install` and `config` inputs were undocumented; the prose measurement was
+stale in English; `data.py` has been per-ecosystem for some time; and the CI and
+pre-commit snippets pinned a version whose own changelog records a false
+all-clear.
+
+656 tests. Field gate: 4,461 packages, zero blocks.
+
+## [0.21.0] - 2026-09-04
+
+Three more false blocks, all found by a new release gate that scans large real
+repositories instead of synthetic manifests. On `home-assistant/core` alone the
+count went **39 -> 0**.
+
+### Added
+- **`scripts/fieldtest.py`, a release gate.** It clones five repositories chosen
+  for the dependency shapes they contain, scans each, and fails on any block
+  not listed with a reason. Every name in those projects is something thousands
+  of people install daily, so **every block is false until shown otherwise**.
+  Current state: **4,461 packages checked, zero blocks.**
+
+  This exists because 605 unit tests and a 35-check acceptance pass had missed
+  eleven defects that one pass over real repositories found immediately.
+
+### Fixed
+- **A constraints file forbids; it does not install.** `pip` never installs
+  from one -- it only bounds a version if the package arrives some other way --
+  so the standard way to forbid a package outright is to pin it to a version
+  that cannot exist. Home Assistant does this for eight of them
+  (`pycrypto==1000000000.0.0`), and checking those pins turned deliberate
+  exclusions into **35 reported blocks**. The flag now survives nested `-c` and
+  `-r` includes. The name in a constraints file is still checked.
+- **PEP 440 pads the release segment with zeros.** `0.8` and `0.8.0` are one
+  version, as are `1.6.6` and `1.6.6.0`. Both spellings sit in Home Assistant
+  requirements against packages that store the other form, and comparing the
+  raw text blocked `libsoundtouch` and `baidu-aip`. Worse, the test suite had
+  asserted the *opposite* -- that `1.0` and `1.0.0` differ -- so the wrong
+  behaviour was locked in by a test. Corrected, and the padding is now
+  asserted.
+- **A block no longer rests on a cached version list.** An established package
+  is cached for a day, so a release published inside that window is invisible;
+  `opower==0.21.0` exists and was blocked from a stale list. The version list
+  is now re-fetched before any version block, which is the same rule that
+  already governs negative answers: the answer that blocks has to be fresh.
+
+614 tests.
+
 ## [0.20.0] - 2026-09-03
 
 Four defects found by scanning **10,109 packages** across `vercel/next.js`,
@@ -810,7 +902,9 @@ First release.
 - No corpus of hallucinated package names is shipped, following the decision of
   the USENIX'25 authors not to publish theirs.
 
-[Unreleased]: https://github.com/M1rwana12/ghostpkg/compare/v0.20.0...HEAD
+[Unreleased]: https://github.com/M1rwana12/ghostpkg/compare/v0.22.0...HEAD
+[0.22.0]: https://github.com/M1rwana12/ghostpkg/releases/tag/v0.22.0
+[0.21.0]: https://github.com/M1rwana12/ghostpkg/releases/tag/v0.21.0
 [0.20.0]: https://github.com/M1rwana12/ghostpkg/releases/tag/v0.20.0
 [0.19.3]: https://github.com/M1rwana12/ghostpkg/releases/tag/v0.19.3
 [0.19.2]: https://github.com/M1rwana12/ghostpkg/releases/tag/v0.19.2

{ghostpkg-0.20.0 → ghostpkg-0.22.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: ghostpkg
-Version: 0.20.0
+Version: 0.22.0
 Summary: Catch package names that do not exist before you install them
 Project-URL: Homepage, https://github.com/m1rwana12/ghostpkg
 Project-URL: Issues, https://github.com/m1rwana12/ghostpkg/issues
@@ -183,8 +183,8 @@ somebody copies the line and runs it — the install has already happened by the
 time that name reaches `requirements.txt`.
 
 Extraction is deliberately narrow, because a README is full of words that look
-like package names. Measured across thirteen real READMEs, it finds the genuine
-names with a **0% false-positive rate**; `pip install -r requirements.txt`,
+like package names. Measured across twenty-two real READMEs it extracted 18
+names, **none of which fail to exist**; `pip install -r requirements.txt`,
 `npm run build`, `pip is a package manager` and `npx create-react-app my-app`
 (where `my-app` is an argument) all yield nothing.
 
@@ -251,7 +251,9 @@ guessed at. Known public mirrors (`registry.yarnpkg.com`,
 
 | Flag | Purpose |
 |---|---|
-| `-e`, `--ecosystem` | `pypi` (default) or `npm` |
+| `-e`, `--ecosystem` | `pypi` (default) or `npm`. `check` only -- `scan` takes the ecosystem from the file |
+| `--format` | `text` (default), `json`, or `github` for diff annotations |
+| `--config PATH` | Ignore file. Never read from the directory being scanned |
 | `--strict` | Promote warnings to blocks |
 | `--json` | Machine-readable output for scripts and CI |
 | `-q`, `--quiet` | Hide packages that passed |
@@ -379,8 +381,8 @@ commands instead, and `--format text` is the default.
 | A pinned version that does not exist | 🔴 **Blocked** | `requests==99.99.99`. A model invents versions as readily as names, and the registry lists every real one, so this is a lookup rather than a heuristic. Only exact pins are checked — a range like `>=2.31` or `^4.18.0` may be satisfied by some other version. |
 | First published < 90 days ago | 🟡 Warning | Attackers register fast. So do honest authors — hence a warning, not a block. |
 | First published < 1 year ago | 🟡 Warning | Weaker version of the same signal. |
-| Only one release | 🟡 Warning | Squats are usually published once and abandoned. |
-| No repository or homepage link | 🟡 Warning | Real projects almost always link to source. |
+| Only one release, **and under a year old** | 🟡 Warning | Squats are usually published once and abandoned. An established package with one release is simply finished, so the age gate is part of the signal rather than a separate row. |
+| No repository or homepage link, **and under a year old** | 🟡 Warning | Real projects almost always link to source. Age-gated for the same reason as the row above. |
 | 1–2 edits from a popular name, **and** the package is either recent **or** abandoned (≤2 releases and no repository link) | 🟡 Warning | Classic typosquat shape. A swap of adjacent characters counts as one edit, because `recat`/`react` is what squatters actually publish. Age alone was the wrong gate: `expresss` has sat on npm since 2016 with one release and ~2,500 typo-driven downloads a month. |
 
 Warnings are advisory by default. Nothing but non-existence blocks unless you pass
@@ -409,9 +411,11 @@ Everything softer is reported for a human to read.
 $ ghostpkg check react-router-dom-utils -e npm
 
   WARNING  react-router-dom-utils
-           - first published 176 days ago
+           - first published 179 days ago (under a year)
            - only one release
            - no repository or homepage link
+
+  1 to review by hand
 ```
 
 When the name does not exist at all, the block comes with the likely intent
@@ -437,7 +441,7 @@ length, and only applies to packages young enough to plausibly be a squat.
 ### In CI
 
 ```yaml
-- uses: M1rwana12/ghostpkg@v0.19.0
+- uses: M1rwana12/ghostpkg@v0.22.0
 ```
 
 That is the whole step. It searches the checkout, skips `node_modules` and
@@ -451,8 +455,11 @@ than leaving the answer in a job log:
 A blocking finding is an error and a soft signal is a warning, so the
 annotations and the exit code agree about severity.
 
-Optional inputs -- `paths`, `strict`, `deep`, `version`, `python-version`,
-`fail-on-error`. Pin `version` if you want the run to be reproducible.
+Optional inputs -- `paths`, `strict`, `deep`, `version`, `install`, `config`,
+`python-version`, `fail-on-error`. Pin `version` for a reproducible run, and
+point `config` at an ignore file kept in the repository (this project uses
+`.github/ghostpkg-ignore.json` for its own scan, so suppressions are reviewed
+in a pull request like any other change).
 
 Or without the action, if you prefer:
 
@@ -465,7 +472,7 @@ Or without the action, if you prefer:
 ```yaml
 repos:
   - repo: https://github.com/M1rwana12/ghostpkg
-    rev: v0.19.0
+    rev: v0.22.0
     hooks:
       - id: ghostpkg
 ```

{ghostpkg-0.20.0 → ghostpkg-0.22.0}/README.en.md

@@ -135,8 +135,8 @@ somebody copies the line and runs it — the install has already happened by the
 time that name reaches `requirements.txt`.
 
 Extraction is deliberately narrow, because a README is full of words that look
-like package names. Measured across thirteen real READMEs, it finds the genuine
-names with a **0% false-positive rate**; `pip install -r requirements.txt`,
+like package names. Measured across twenty-two real READMEs it extracted 18
+names, **none of which fail to exist**; `pip install -r requirements.txt`,
 `npm run build`, `pip is a package manager` and `npx create-react-app my-app`
 (where `my-app` is an argument) all yield nothing.
 
@@ -203,7 +203,9 @@ guessed at. Known public mirrors (`registry.yarnpkg.com`,
 
 | Flag | Purpose |
 |---|---|
-| `-e`, `--ecosystem` | `pypi` (default) or `npm` |
+| `-e`, `--ecosystem` | `pypi` (default) or `npm`. `check` only -- `scan` takes the ecosystem from the file |
+| `--format` | `text` (default), `json`, or `github` for diff annotations |
+| `--config PATH` | Ignore file. Never read from the directory being scanned |
 | `--strict` | Promote warnings to blocks |
 | `--json` | Machine-readable output for scripts and CI |
 | `-q`, `--quiet` | Hide packages that passed |
@@ -331,8 +333,8 @@ commands instead, and `--format text` is the default.
 | A pinned version that does not exist | 🔴 **Blocked** | `requests==99.99.99`. A model invents versions as readily as names, and the registry lists every real one, so this is a lookup rather than a heuristic. Only exact pins are checked — a range like `>=2.31` or `^4.18.0` may be satisfied by some other version. |
 | First published < 90 days ago | 🟡 Warning | Attackers register fast. So do honest authors — hence a warning, not a block. |
 | First published < 1 year ago | 🟡 Warning | Weaker version of the same signal. |
-| Only one release | 🟡 Warning | Squats are usually published once and abandoned. |
-| No repository or homepage link | 🟡 Warning | Real projects almost always link to source. |
+| Only one release, **and under a year old** | 🟡 Warning | Squats are usually published once and abandoned. An established package with one release is simply finished, so the age gate is part of the signal rather than a separate row. |
+| No repository or homepage link, **and under a year old** | 🟡 Warning | Real projects almost always link to source. Age-gated for the same reason as the row above. |
 | 1–2 edits from a popular name, **and** the package is either recent **or** abandoned (≤2 releases and no repository link) | 🟡 Warning | Classic typosquat shape. A swap of adjacent characters counts as one edit, because `recat`/`react` is what squatters actually publish. Age alone was the wrong gate: `expresss` has sat on npm since 2016 with one release and ~2,500 typo-driven downloads a month. |
 
 Warnings are advisory by default. Nothing but non-existence blocks unless you pass
@@ -361,9 +363,11 @@ Everything softer is reported for a human to read.
 $ ghostpkg check react-router-dom-utils -e npm
 
   WARNING  react-router-dom-utils
-           - first published 176 days ago
+           - first published 179 days ago (under a year)
            - only one release
            - no repository or homepage link
+
+  1 to review by hand
 ```
 
 When the name does not exist at all, the block comes with the likely intent
@@ -389,7 +393,7 @@ length, and only applies to packages young enough to plausibly be a squat.
 ### In CI
 
 ```yaml
-- uses: M1rwana12/ghostpkg@v0.19.0
+- uses: M1rwana12/ghostpkg@v0.22.0
 ```
 
 That is the whole step. It searches the checkout, skips `node_modules` and
@@ -403,8 +407,11 @@ than leaving the answer in a job log:
 A blocking finding is an error and a soft signal is a warning, so the
 annotations and the exit code agree about severity.
 
-Optional inputs -- `paths`, `strict`, `deep`, `version`, `python-version`,
-`fail-on-error`. Pin `version` if you want the run to be reproducible.
+Optional inputs -- `paths`, `strict`, `deep`, `version`, `install`, `config`,
+`python-version`, `fail-on-error`. Pin `version` for a reproducible run, and
+point `config` at an ignore file kept in the repository (this project uses
+`.github/ghostpkg-ignore.json` for its own scan, so suppressions are reviewed
+in a pull request like any other change).
 
 Or without the action, if you prefer:
 
@@ -417,7 +424,7 @@ Or without the action, if you prefer:
 ```yaml
 repos:
   - repo: https://github.com/M1rwana12/ghostpkg
-    rev: v0.19.0
+    rev: v0.22.0
     hooks:
       - id: ghostpkg
 ```

{ghostpkg-0.20.0 → ghostpkg-0.22.0}/README.md

@@ -98,7 +98,9 @@ ghostpkg clear-cache
 
 | Прапорець | Призначення |
 |---|---|
-| `-e`, `--ecosystem` | `pypi` (типово) або `npm` |
+| `-e`, `--ecosystem` | `pypi` (типово) або `npm`. Лише для `check` — `scan` визначає екосистему з файлу |
+| `--format` | `text` (типово), `json`, або `github` для анотацій на діф |
+| `--config PATH` | Ignore-файл. Ніколи не читається з каталогу, який сканують |
 | `--strict` | Підвищує попередження до блокувань |
 | `--json` | Вивід у JSON для скриптів і CI |
 | `-q`, `--quiet` | Ховає пакети, які пройшли перевірку |
@@ -117,7 +119,10 @@ Lock-файли варто перевіряти навіть якщо ви вж
 включно з транзитивними, яких у маніфесті немає. І кожен запис має точну версію,
 отже перевіряється повністю.
 
-**Читає й прозу** — `README`, `AGENTS.md`, `CLAUDE.md`, `.cursorrules`, будь-який `.md`.
+**Читає й прозу** — `README`, `AGENTS.md`, `CLAUDE.md`, `.cursorrules`,
+`.windsurfrules`, і будь-який `.md`, названий прямо. При пошуку в каталозі
+беруться лише файли інструкцій для агентів та README **у корені** — інакше
+великий репозиторій завалив би вивід чейнджлогами.
 Це закриває проблему порядку: **галюцинація з'являється раніше за маніфест**. Модель
 пише `pip install foo-bar` у README, людина копіює рядок — і встановлення вже відбулось,
 коли назва тільки потрапляє в `requirements.txt`.
@@ -185,7 +190,7 @@ ghostpkg scan ../інший-проєкт
 ### У CI
 
 ```yaml
-- uses: M1rwana12/ghostpkg@v0.19.0
+- uses: M1rwana12/ghostpkg@v0.22.0
 ```
 
 Це весь крок. Він шукає маніфести в чекауті, обходить `node_modules` і подібні
@@ -199,15 +204,17 @@ ghostpkg scan ../інший-проєкт
 Блокувальна знахідка — помилка, м'який сигнал — попередження, тож анотації й код
 виходу не суперечать одне одному.
 
-Необов'язкові входи: `paths`, `strict`, `deep`, `version`, `python-version`,
-`fail-on-error`.
+Необов'язкові входи: `paths`, `strict`, `deep`, `version`, `install`, `config`,
+`python-version`, `fail-on-error`. `config` вказує на ignore-файл у самому
+репозиторії — цей проєкт тримає свій у `.github/ghostpkg-ignore.json`, тож
+придушення проходить рев'ю як звичайна зміна.
 
 ### Як хук pre-commit
 
 ```yaml
 repos:
   - repo: https://github.com/M1rwana12/ghostpkg
-    rev: v0.19.0
+    rev: v0.22.0
     hooks:
       - id: ghostpkg
 ```
@@ -227,8 +234,8 @@ repos:
 | Закріплену версію відкликав мейнтейнер | 🟡 Попередження з його ж поясненням. Не блокування: `pip` встановлює відкликану версію, якщо її закріпили явно. |
 | Закріплена версія не існує (`requests==99.99.99`) | 🔴 **Заблоковано.** Вигадана версія — той самий клас помилки, що й вигадана назва. Діапазони (`>=2.31`, `^4.18`) не перевіряються: їх може задовольнити інша версія. |
 | Опубліковано днями тому | 🟡 Попередження. Зловмисники реєструють швидко — але й чесні автори теж. |
-| Лише один реліз | 🟡 Попередження. |
-| Немає посилання на репозиторій | 🟡 Попередження. |
+| Лише один реліз, **і пакету менше року** | 🟡 Попередження. Усталений пакет з одним релізом — просто завершений, тож віковий фільтр тут частина сигналу. |
+| Немає посилання на репозиторій, **і пакету менше року** | 🟡 Попередження. Віковий фільтр із тієї ж причини, що й рядком вище. |
 | За один-два символи від популярної назви, **і** пакет свіжий **або** занедбаний (≤2 релізи й немає репозиторію) | 🟡 Попередження. Схоже на typosquat. |
 
 ---

{ghostpkg-0.20.0 → ghostpkg-0.22.0}/ghostpkg/__init__.py

@@ -1,6 +1,6 @@
 """ghostpkg -- catch package names that do not exist before you install them."""
 
-__version__ = "0.20.0"
+__version__ = "0.22.0"
 
 from .assess import Finding, Verdict, assess
 from .registries import PackageFacts, RegistryError, fetch

{ghostpkg-0.20.0 → ghostpkg-0.22.0}/ghostpkg/cli.py

@@ -13,6 +13,7 @@ from pathlib import Path
 
 from . import __version__, registries
 from .assess import Finding
+from . import inspection
 from .cache import Cache
 from .discover import discover
 from .manifests import (
@@ -120,8 +121,17 @@ def main(argv: list[str] | None = None) -> int:
         print(f"ghostpkg: {'removed ' + str(cache.path) if removed else 'nothing to remove'}")
         return EXIT_OK
 
-    if args.timeout:
+    if args.timeout is not None:
+        # `if args.timeout:` made `--timeout 0` a silent no-op. Nought is not a
+        # sensible budget either, so it is refused rather than ignored.
+        if args.timeout <= 0:
+            print("ghostpkg: --timeout must be a positive number of seconds", file=sys.stderr)
+            return EXIT_ERROR
         registries.TIMEOUT = args.timeout
+        # `--deep` downloads an archive through its own client, which had a
+        # fixed budget of its own -- so the documented per-request timeout
+        # applied to every request except the slowest kind.
+        inspection.TIMEOUT = max(args.timeout, inspection.TIMEOUT)
 
     # Group by ecosystem so several manifests are looked up together and a
     # dependency repeated across them costs one request, not one per file.

{ghostpkg-0.20.0 → ghostpkg-0.22.0}/ghostpkg/jslocks.py

@@ -49,6 +49,35 @@ PNPM_LOCAL = ("file:", "link:", "http://", "https://", "git+", "git:")
 PNPM_VERSION_AT = re.compile(r"(?<=.)@(?=\d)")
 
 
+def _pnpm_name(key: str) -> "str | None":
+    """The package name out of a `packages:` key, whichever version wrote it.
+
+        /react-dom/18.2.0_react@18.2.0     v5, version after a slash
+        /react@18.2.0                      v6, version after an @
+        'react@18.2.0'                     v9, the same with quotes
+
+    The shapes are told apart by structure rather than by hunting for the
+    first `@` before a digit. That hunt was wrong for every v5 key carrying a
+    peer suffix, because the suffix contains an `@digit` of its own: it read
+    `/react-dom/18.2.0_react@18.2.0` as a package called
+    `react-dom/18.2.0_react`, and did the same to `eslint-plugin-react` and
+    `@babel/helper-compilation-targets` -- 17 false blocks in one repository,
+    on some of the most-installed packages there are.
+
+    In a v5 key the last slash-separated segment is the version, so it starts
+    with a digit. In a scoped v6 key the last segment is `name@version` and
+    starts with a letter. That single test separates them.
+    """
+    tail = key.rsplit("/", 1)
+    if len(tail) == 2 and tail[1][:1].isdigit() and not key.startswith("@" + tail[1]):
+        return tail[0] or None
+
+    match = PNPM_VERSION_AT.search(key)
+    if match:
+        return key[: match.start()] or None
+    return None
+
+
 def _is_host_path(key: str) -> bool:
     """`github.com/acme/forked/abc123` -- a v5 git dependency, keyed by host.
 
@@ -164,22 +193,20 @@ def parse_pnpm_lock(text: str, source: str | None = None) -> list[Requirement]:
         key = key.split("(", 1)[0]  # v9 peer resolutions
         if key.startswith("/"):
             key = key[1:]
-        # The protocol is only a prefix in v5. From v6 it follows `name@`, so
-        # testing the start of the key never fired and the parser fell through
-        # to its slash split -- emitting names like `github.com/acme/forked`,
-        # which were then looked up on npmjs and blocked.
-        if key.startswith(PNPM_LOCAL) or "://" in key or _is_host_path(key):
-            continue
 
-        match = PNPM_VERSION_AT.search(key)
-        if match:
-            name = key[: match.start()]
-        elif "/" in key.lstrip("@"):
-            # v5 keys the version with a slash: @babel/code-frame/7.12.11
-            name = key.rsplit("/", 1)[0]
-        else:
+        # A protocol can sit anywhere in the key, not only at the front. v5
+        # writes `/file:packages/ui`, v6 and later write
+        # `name@file:packages/ui`, and testing only the start left every
+        # modern workspace with local test fixtures being looked up on npmjs:
+        # `e2e-test-dep-plain@file:...` in SvelteKit, `@test/...-fake-adapter`
+        # in Astro. A `:` cannot appear in a published npm name, so finding
+        # one of these anywhere is decisive.
+        if any(marker in key for marker in PNPM_LOCAL) or "://" in key:
+            continue
+        if _is_host_path(key):
             continue
 
+        name = _pnpm_name(key)
         if name and name not in seen:
             seen.add(name)
             found.append(Requirement(name=name, source=source))

{ghostpkg-0.20.0 → ghostpkg-0.22.0}/ghostpkg/manifests.py

@@ -50,6 +50,46 @@ class Requirement:
     #: README can hold `pip install x` and `npm i y` in adjacent lines, so the
     #: ecosystem cannot always come from the file.
     ecosystem: str | None = None
+    #: From a constraints file, reached by `-c` or named `constraints`. Pip
+    #: never installs from one -- it only bounds a version if the package
+    #: arrives some other way -- so a pin there is a rule, not a request. The
+    #: standard way to forbid a package outright is to pin it to a version that
+    #: cannot exist, and Home Assistant does exactly that for eight of them:
+    #: `pycrypto==1000000000.0.0`. Checking those pins reported a deliberate
+    #: exclusion as a version that does not exist.
+    constraint: bool = False
+
+
+#: A bare digest -- MD5, SHA-1, SHA-256. Airflow keeps 127 files under
+#: `dev/breeze/doc/images/` whose entire content is one checksum, and five of
+#: them carry `constraints` or `requirements` in the name. A 32-character hex
+#: string is a perfectly legal PEP 508 name, so it was read as a package and
+#: blocked for not existing.
+DIGEST = re.compile(r"^[0-9a-f]{32}$|^[0-9a-f]{40}$|^[0-9a-f]{64}$", re.I)
+
+#: A name that is really a file name. Git materialises a symlink as a plain
+#: file containing its target on a Windows checkout without symlink support, so
+#: Ray's `requirements_compiled_py3.10.txt` held the single line
+#: `requirements_compiled.txt` -- read as a package, and blocked.
+#:
+#: Only the two extensions that requirements files themselves use. A wider list
+#: looked tidier and was wrong: `ruamel.yaml`, `ruamel.yaml.clib` and
+#: `pytest.ini` are all real packages, and rejecting by extension would have
+#: turned a false block into a silent miss on three of them.
+FILE_SUFFIXES = (".txt", ".in")
+
+
+def _is_not_a_package(name: str, specifier: "str | None") -> bool:
+    """Both shapes are a whole line on its own, with no version after it.
+
+    That matters: a package really named like a checksum, pinned to a version,
+    is still a requirement. The guard is for a line that carries nothing but
+    the token -- which is what a checksum file and a materialised symlink both
+    look like.
+    """
+    if specifier:
+        return False
+    return bool(DIGEST.match(name)) or name.lower().endswith(FILE_SUFFIXES)
 
 
 # `name @ https://...` is a direct reference: the source is stated explicitly
@@ -67,9 +107,12 @@ NPM_LOCAL_PREFIXES = (
     "github:", "gitlab:", "bitbucket:", "gist:",
 )
 
-#: A version range never starts with a letter, which is how `npm:lodash` is
-#: told apart from `npm:^4.17.19`.
+#: A version range never starts with a letter -- except for the `v` some
+#: people write before a number. That exception cost two false blocks in
+#: Storybook's lockfile, where `pino-abstract-transport@npm:v1.1.0` was read as
+#: an alias to a package called `v1.1.0`.
 RANGE_START = "^~><=*0123456789 "
+_V_RANGE = re.compile(r"^v\d")
 
 #: `"dep": "owner/repo"` / `"owner/repo#semver:^1"` is GitHub shorthand. A
 #: version range never contains a slash, so this does not catch one.
@@ -142,7 +185,9 @@ def npm_alias_target(spec: str) -> "str | None":
     at = rest.rfind("@")
     if at > 0:
         return rest[:at]
-    return None if rest[0] in RANGE_START else rest
+    if rest[0] in RANGE_START or _V_RANGE.match(rest):
+        return None
+    return rest
 
 
 def npm_target(name: str, spec: "str | None") -> "str | None":
@@ -170,6 +215,7 @@ def parse_requirements(
     *,
     base: Path | None = None,
     source: str | None = None,
+    constraint: bool = False,
     _seen: set[Path] | None = None,
     _depth: int = 0,
 ) -> list[Requirement]:
@@ -196,7 +242,11 @@ def parse_requirements(
         if line.startswith("-"):
             include = _include_target(line)
             if include and base is not None and _depth < MAX_INCLUDE_DEPTH:
-                found.extend(_read_include(include, base, seen, _depth))
+                # `-c` names a constraints file; `-r` another requirements one.
+                # The distinction survives the include, because everything a
+                # constraints file lists is a bound rather than an install.
+                nested = constraint or line.lstrip("-").startswith(("c", "-constraint"))
+                found.extend(_read_include(include, base, seen, _depth, nested))
             continue
 
         # A bare URL or local path, not a name we can look up.
@@ -207,13 +257,16 @@ def parse_requirements(
             continue
 
         match = NAME.match(line)
-        if match:
+        if match and not _is_not_a_package(
+            match.group(1), _specifier(line[match.end() :])
+        ):
             found.append(
                 Requirement(
                     name=match.group(1),
                     specifier=_specifier(line[match.end() :]),
                     line=number,
                     source=source,
+                    constraint=constraint,
                 )
             )
 
@@ -240,7 +293,7 @@ def _include_target(line: str) -> str | None:
 
 
 def _read_include(
-    target: str, base: Path, seen: set[Path], depth: int
+    target: str, base: Path, seen: set[Path], depth: int, constraint: bool = False
 ) -> list[Requirement]:
     if _is_url(target):
         return []
@@ -256,7 +309,12 @@ def _read_include(
     except (OSError, UnicodeDecodeError):
         return []
     return parse_requirements(
-        text, base=path.parent, source=str(path), _seen=seen, _depth=depth + 1
+        text,
+        base=path.parent,
+        source=str(path),
+        constraint=constraint or "constraint" in path.name.lower(),
+        _seen=seen,
+        _depth=depth + 1,
     )
 
 
@@ -759,7 +817,15 @@ def load_manifest(path: Path) -> tuple[list[Requirement], str]:
         return parse_pyproject(read_source(path), source), "pypi"
     if _looks_like_requirements(name):
         text = read_source(path)
-        return parse_requirements(text, base=path.parent, source=source), "pypi"
+        return (
+            parse_requirements(
+                text,
+                base=path.parent,
+                source=source,
+                constraint="constraint" in name,
+            ),
+            "pypi",
+        )
 
     raise UnsupportedManifest(
         f"don't know how to read {path.name!r}. Supported: {SUPPORTED}"

{ghostpkg-0.20.0 → ghostpkg-0.22.0}/ghostpkg/registries.py

@@ -93,16 +93,28 @@ _SEGMENT = re.compile(r"(\d+|[a-z]+)")
 
 
 def normalise_version(version: str) -> str:
-    """A PyPI version in the form PyPI itself stores.
-
-    Deliberately narrow: it folds case, a leading `v`, the `-`/`_` separators
-    and leading zeros in numeric segments, and the pre-release spellings. It is
-    not a full PEP 440 implementation and does not order versions -- it only
-    has to decide whether two spellings name the same release.
+    """A PyPI version reduced to the release it names.
+
+    PEP 440 compares release segments by padding the shorter one with zeros, so
+    `0.8` and `0.8.0` are one version and `1.6.6` and `1.6.6.0` are another.
+    Both spellings sit in unmodified Home Assistant requirements against
+    packages that store the other form, and comparing the raw text blocked
+    them as versions that do not exist.
+
+    Deliberately narrow: it folds case, a leading `v`, the `-`/`_` separators,
+    leading zeros, trailing zero components of the release, and the
+    pre-release spellings. It is not a full PEP 440 implementation and does not
+    order versions -- it only decides whether two spellings name one release.
     """
     text = version.strip().lower().lstrip("v")
+    # A local version identifier -- `2.9.0+cu128` -- names a build made
+    # somewhere else. PyPI refuses uploads that carry one, so it can never
+    # appear in a release list, and comparing with it attached blocked every
+    # CUDA-pinned torch line in Ray.
+    text = text.split("+", 1)[0]
     for separator in ("-", "_"):
         text = text.replace(separator, ".")
+
     parts = []
     for segment in text.split("."):
         for token in _SEGMENT.findall(segment) or [segment]:
@@ -110,7 +122,18 @@ def normalise_version(version: str) -> str:
                 parts.append(str(int(token)))
             else:
                 parts.append(_PRE_SPELLINGS.get(token, token))
-    return ".".join(p for p in parts if p)
+    parts = [p for p in parts if p]
+
+    # Trailing zeros are dropped from the release segment only. Everything
+    # from the first non-numeric token onwards is a pre/post/dev marker, where
+    # a zero is meaningful -- `1.0rc0` is not `1.0rc`.
+    release = 0
+    while release < len(parts) and parts[release].isdigit():
+        release += 1
+    head, tail = parts[:release], parts[release:]
+    while len(head) > 1 and head[-1] == "0":
+        head.pop()
+    return ".".join(head + tail)
 
 
 def _get_json(url: str, timeout: int | None = None) -> dict | None: