@thomasminh1995/depverdict 0.6.0-alpha.1

package/docs/MVP-02-Knowledge-Manifest-Generation.md

@@ -0,0 +1,75 @@
+# MVP-02 Knowledge Manifest Generation and CLI
+
+MVP-02-08 turns the validated internal `KnowledgeResearchResult` into the portable Knowledge Manifest consumed by MVP-03. The public artifact is written only after JSON Schema and runtime invariant validation pass.
+
+```text
+Project Manifest → Research Plan → Knowledge Research
+                                      │
+                                      ▼
+                         KnowledgeResearchResult
+                                      │
+                                      ▼
+                      Knowledge Manifest builder
+                                      │
+                                      ▼
+                    schema + runtime validation
+                                      │
+                                      ▼
+                 atomic knowledge-manifest.json write
+```
+
+## Builder and public contract
+
+The builder creates Knowledge Manifest schema `1.0.0` without mutating the internal research result. It preserves package and source IDs, occurrences, registry facts, provenance, warnings, execution timestamps, and public cache totals. Internal execution-only fields such as adapter invocation detail, cache corruption bookkeeping, invalid occurrence evidence, unsupported ecosystem aggregates, and source-candidate field paths do not enter the public artifact.
+
+Research occurrences contain internal ecosystem and normalized-name fields while planning and orchestration run. The builder projects them onto the public occurrence contract, which retains project path, manifest path, dependency type, declared name, and declared version only.
+
+The builder validates with the existing Draft 2020-12 [Knowledge Manifest schema](../schemas/knowledge-manifest.schema.json) and `validateKnowledgeManifestInvariants`. Invalid input throws before any writer is called.
+
+## Deterministic research identity
+
+`research.researchId` is `sha256` over canonical JSON containing:
+
+1. the exact Project Manifest artifact digest;
+2. a canonical fingerprint of the public research policy; and
+3. source digests ordered by source ID.
+
+A source uses its snapshot content digest when available. An unfetched source has a digest of its canonical public provenance fields instead. Timestamps, temporary filenames, adapter completion order, and random values do not affect the ID.
+
+## Atomic serialization
+
+The writer uses UTF-8, two-space JSON indentation, and one final newline. It creates the parent directory, writes a sibling temporary file, flushes it, closes it, then renames it over the destination. Existing manifests are replaced atomically; a failed write never exposes a partial target file.
+
+The default artifact location is:
+
+```text
+.upgradelens/knowledge-manifest.json
+```
+
+## CLI
+
+Run discovery first, then research the generated Project Manifest:
+
+```bash
+upgradelens discover .
+upgradelens research .
+```
+
+`research` supports:
+
+```bash
+upgradelens research .
+upgradelens research . --output artifacts/knowledge.json
+upgradelens research . --stdout
+upgradelens research . --offline
+```
+
+`--stdout` emits only pretty Knowledge Manifest JSON. Normal execution reports concise progress to standard error. `--output` is a portable path relative to the repository root.
+
+## Offline mode
+
+`--offline` configures npm and PyPI adapters for fresh-cache-only research. No registry request is attempted. Fresh cache entries normalize normally. Missing, expired, or corrupted entries do not fall back to networking or stale data; the package becomes `unavailable` with an `OFFLINE_CACHE_MISS` warning. Existing cache details, paths, keys, headers, validators, and response bodies remain private.
+
+## Boundary
+
+MVP-02-08 writes the public Knowledge Manifest but does not implement version comparison, release analysis, GitHub APIs, documentation crawling, source-content fetching, breaking-change analysis, impact analysis, migration planning, AI, or a new analysis artifact. MVP-03 consumes the immutable manifest as its input.

package/docs/MVP-02-Knowledge-Manifest.md

@@ -0,0 +1,279 @@
+# MVP-02 Knowledge Manifest Contract
+
+The Knowledge Manifest is the versioned public artifact produced by UpgradeLens Knowledge Research. It captures normalized package facts, source provenance, execution metadata, cache outcomes, and non-fatal warnings for one Project Manifest. MVP-02 collects facts; it does not compare versions, identify breaking changes, assess repository impact, or recommend migrations.
+
+The contract is defined by the [Knowledge Manifest JSON Schema](../schemas/knowledge-manifest.schema.json). Schema version `1.0.0` is independent from Project Manifest schema `2.0.0`, the npm package version, and later analysis-artifact versions.
+
+## Ownership and immutability
+
+MVP-02 owns the Knowledge Manifest. Once a manifest has passed schema and runtime invariant validation and has been published atomically, it is immutable. Downstream stages consume it without repairing or enriching it in place:
+
+```text
+Project Manifest (MVP-01)
+        │
+        ▼
+Knowledge Manifest (MVP-02)
+        │
+        ▼
+Version Analysis Manifest (MVP-03)
+```
+
+If an upstream fact is missing, the owning stage must produce a new versioned artifact. MVP-03 may not modify the Knowledge Manifest or rescan the repository.
+
+## Knowledge Store versus Knowledge Manifest
+
+The Knowledge Store and Knowledge Manifest serve different purposes:
+
+| Knowledge Store | Knowledge Manifest |
+| --- | --- |
+| Internal reusable cache | Portable public artifact |
+| Content-addressed response envelopes | Normalized facts and provenance |
+| Owns ETag, Last-Modified, TTL state, cache keys, and storage layout | Owns portable content digests, source freshness, and cache counts |
+| May be deleted and rebuilt | Immutable after successful publication |
+| Not consumed as an MVP-03 contract | Consumed directly by downstream MVPs |
+
+The manifest rejects store paths, internal cache keys, HTTP validators, authorization data, and unversioned storage-envelope fields through `additionalProperties: false`. It also rejects local absolute paths. Content digests identify evidence without exposing how or where it is stored.
+
+## Top-level contract
+
+Every Knowledge Manifest requires these fields:
+
+| Field | Purpose |
+| --- | --- |
+| `schemaVersion` | Knowledge Manifest contract version; exactly `1.0.0` |
+| `generatedAt` | UTC date-time when the validated artifact was assembled |
+| `generator` | UpgradeLens name and producing package version |
+| `input` | Project Manifest lineage without duplicating the input artifact |
+| `policy` | Non-secret research behavior for this execution |
+| `research` | Execution identity, timestamps, counts, retries, and partial failures |
+| `summary` | Canonical package/source/status/warning/cache totals |
+| `packages` | Package-centric research records |
+| `sources` | Deduplicated source and snapshot provenance records |
+| `cache` | Portable cache outcome summary |
+| `warnings` | Deterministically ordered non-fatal conditions |
+
+All public objects reject undeclared fields. The schema document itself has no custom top-level `schemaVersion` keyword; the version is expressed only as `properties.schemaVersion.const`.
+
+## Generator and schema versioning
+
+`generator.name` is always `UpgradeLens`. `generator.version` records the package version that produced the artifact but is intentionally not a schema constant. A package release can continue producing Knowledge Manifest `1.0.0` while implementation details evolve compatibly.
+
+An incompatible public-contract change requires a new Knowledge Manifest schema version. Project Manifest schema changes independently. Consumers must validate the version they support before reading nested records.
+
+## Project Manifest lineage
+
+`input.projectManifest` records:
+
+- Project Manifest schema version `2.0.0`;
+- its repository-relative artifact path;
+- a `sha256:<64 lowercase hexadecimal characters>` digest;
+- repository name and root `.`.
+
+`artifactDigest` fingerprints the exact bytes of the Project Manifest, not a reserialized object. This preserves lineage without copying the complete Project Manifest into the Knowledge Manifest. Artifact, project, and dependency-manifest paths use portable forward-slash relative paths; absolute paths, Windows drive paths, backslashes, and parent traversal are rejected.
+
+## Research policy
+
+`policy` records execution behavior rather than an upgrade recommendation:
+
+- `mode`: `online` or `offline`;
+- `policyVersion`;
+- public HTTPS `registryBases` for npm and PyPI;
+- `ttlPolicyVersion`;
+- `sourceAllowlistVersion`;
+- `includePrereleases`.
+
+Only public npm and PyPI bases exist in the MVP-02 contract. Tokens, authorization headers, signed URLs, credentials, and arbitrary secret fields are not allowed.
+
+## Research execution metadata
+
+`research` contains a deterministic `researchId`, `startedAt`, `completedAt`, derived `durationMs`, input occurrence/package counts, researched package and source counts, cache hit/miss/revalidation counts, retry count, and partial-failure count.
+
+The research ID is a SHA-256 identity derived from the input artifact digest, policy fingerprint, and canonically ordered source content digests. It is not a transient process/run ID. Timestamps and duration are execution facts and can differ between executions even when normalized package facts are unchanged.
+
+Runtime invariants define:
+
+- `inputPackageCount` as the number of package records;
+- `researchedPackageCount` as package records excluding `invalid` identities;
+- `partialFailureCount` as package records whose status is not `resolved`;
+- `durationMs` as `completedAt - startedAt` in milliseconds.
+
+## Summary and relational invariants
+
+The summary reports input occurrences, unique package records, every package-status count, sources, warnings, cache hits/misses, and stale sources. These are counts of known manifest structures, not placeholders for unavailable external facts.
+
+JSON Schema validates field presence, type, and non-negative integer ranges. The focused validator in `src/knowledge-manifest.js` validates relationships JSON Schema cannot express cleanly:
+
+- package status counts sum to `packageCount`;
+- package, occurrence, source, warning, and stale-source counts match their arrays;
+- research and cache counts agree with the summary;
+- duration agrees with execution timestamps;
+- package/source IDs are unique and references resolve;
+- package IDs agree with their registry and normalized identity;
+- all contract arrays use canonical ordering.
+
+Writers must pass both layers before atomically publishing the artifact.
+
+## Package identity and status
+
+MVP-02 package IDs are package-centric:
+
+```text
+npm:<normalized-name>
+pypi:<normalized-name>
+```
+
+Scoped npm IDs such as `npm:@scope/package` are valid. `identity.observedDeclaredNames` preserves declared spelling from the Project Manifest, while `normalizedName` is the registry lookup identity. `registryBaseUrl`, canonical package page, and API URL are HTTPS.
+
+Package status is one of:
+
+- `resolved`: required registry facts are available;
+- `partial`: useful facts exist but one or more sources failed, conflicted, or are stale;
+- `notFound`: the authoritative registry confirmed no package;
+- `invalid`: the Project Manifest reference cannot form a supported public identity;
+- `unavailable`: research could not obtain usable facts because a source was unavailable.
+
+Every package contains `metadata`, `latest`, and `releaseIndex`. Missing optional metadata is omitted or represented by `null`; it is never represented by fake counts or invented values. `notFound`, `invalid`, and `unavailable` packages require `latest: null` and an empty release index, preventing unresolved facts from masquerading as valid zero values.
+
+## Occurrences
+
+Package deduplication never discards Project Manifest occurrences. Each occurrence retains:
+
+- project ID and repository-relative project path;
+- repository-relative dependency manifest path;
+- dependency type;
+- declared name;
+- declared version/reference, or `null` when unversioned.
+
+Multiple projects can reference one package record, and duplicate declarations remain separate occurrences. This preserves repository context while keeping external knowledge package-centric.
+
+## Registry-designated latest and releases
+
+`latest` records a registry fact, not a recommended target. npm uses `dist-tag:latest`; PyPI uses `project-info-version`. MVP-02 does not compute a maximum version or interpret npm SemVer or PEP 440.
+
+Latest facts retain version, selection mechanism, optional publication/release URL data, source ID, and nullable prerelease/yanked/deprecated flags. `null` means the source did not provide that fact; `false` means it explicitly reported false.
+
+Release entries contain either an opaque `version` or opaque `tag`, optional publication time/URL, nullable status flags, and source IDs. MVP-02 sorts identifiers lexically and performs no semantic ordering.
+
+## Source authority, trust, and snapshots
+
+Each source has a stable ID, kind, field-specific authority, categorical trust, canonical HTTPS URL, status, supported evidence roles, discovery provenance, trust evidence, and an optional portable snapshot.
+
+Source kinds are registry, official documentation, source repository, release feed, or community. Authority is `registryAuthoritative`, `officialProject`, `publisherProvided`, or `community`. Trust is categorical—`official`, `publisher`, `verified`, `community`, or `unknown`—and has no numeric score.
+
+Registry records are authoritative for registry publication facts, but descriptions and links inside them remain publisher-controlled. Official documentation is preferred for project policy and usage statements. Community sources never override official or registry-authoritative facts.
+
+A snapshot contains only content digest, media type, retrieval time, and `fresh`/`stale` state. `discoveredFrom`, `trustEvidenceSourceIds`, and `conflictsWith` reference top-level source IDs and are checked at runtime.
+
+## Warning behavior
+
+Warnings represent non-fatal research conditions:
+
+- `INVALID_PACKAGE_REFERENCE`
+- `PACKAGE_NOT_FOUND`
+- `REGISTRY_UNAVAILABLE`
+- `REGISTRY_RATE_LIMITED`
+- `REGISTRY_RESPONSE_INVALID`
+- `DOCUMENTATION_NOT_FOUND`
+- `REPOSITORY_NOT_FOUND`
+- `RELEASE_METADATA_NOT_FOUND`
+- `CACHE_EXPIRED`
+- `OFFLINE_CACHE_MISS`
+- `SOURCE_CONFLICT`
+
+Each warning has a deterministic sanitized message and `retryable` flag. Package/source IDs are present when the condition can be scoped. A package stores warning-code references only; full records remain at the top level. Runtime validation checks both directions of that relationship.
+
+Warnings never contain stack traces, secrets, absolute paths, raw headers, or complete upstream error bodies. A warning does not by itself make the artifact invalid; package and source statuses expose the resulting completeness.
+
+## Cache summary
+
+The public `cache` object includes only mode, TTL policy version, hit/miss/revalidation counts, and stale-entry count. Counts must agree with `research` and `summary`. Store paths, keys, validators, envelopes, and persistence APIs remain private to the Knowledge Store.
+
+## Deterministic ordering
+
+Writers use code-unit lexical comparison, not locale-sensitive comparison:
+
+1. packages by `id`;
+2. occurrences by project ID, manifest, dependency type, declared name, and declared version;
+3. releases by opaque version/tag, then joined source IDs;
+4. sources by `id`;
+5. observed names, source ID arrays, warning codes, evidence roles, trust evidence, and conflict IDs lexically;
+6. warnings by package ID, code, source ID, and message.
+
+JSON Schema cannot assert array ordering or referential integrity. The runtime invariant validator rejects unsorted output and unknown references, preventing asynchronous completion or source payload enumeration from affecting the artifact.
+
+## Example
+
+This empty but complete manifest is valid:
+
+```json
+{
+  "schemaVersion": "1.0.0",
+  "generatedAt": "2026-07-14T00:00:00.000Z",
+  "generator": { "name": "UpgradeLens", "version": "<package version>" },
+  "input": {
+    "projectManifest": {
+      "schemaVersion": "2.0.0",
+      "artifact": ".upgradelens/project-manifest.json",
+      "artifactDigest": "sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
+      "repository": { "name": "example-project", "root": "." }
+    }
+  },
+  "policy": {
+    "mode": "online",
+    "policyVersion": "1",
+    "registryBases": {
+      "npm": "https://registry.npmjs.org",
+      "pypi": "https://pypi.org"
+    },
+    "ttlPolicyVersion": "1",
+    "sourceAllowlistVersion": "1",
+    "includePrereleases": false
+  },
+  "research": {
+    "researchId": "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
+    "startedAt": "2026-07-14T00:00:00.000Z",
+    "completedAt": "2026-07-14T00:00:00.000Z",
+    "durationMs": 0,
+    "inputOccurrenceCount": 0,
+    "inputPackageCount": 0,
+    "researchedPackageCount": 0,
+    "sourceCount": 0,
+    "cacheHitCount": 0,
+    "cacheMissCount": 0,
+    "cacheRevalidationCount": 0,
+    "retryCount": 0,
+    "partialFailureCount": 0
+  },
+  "summary": {
+    "inputOccurrenceCount": 0,
+    "packageCount": 0,
+    "resolvedPackageCount": 0,
+    "partialPackageCount": 0,
+    "notFoundPackageCount": 0,
+    "invalidPackageCount": 0,
+    "unavailablePackageCount": 0,
+    "sourceCount": 0,
+    "warningCount": 0,
+    "cacheHitCount": 0,
+    "cacheMissCount": 0,
+    "staleSourceCount": 0
+  },
+  "packages": [],
+  "sources": [],
+  "cache": {
+    "mode": "online",
+    "policyVersion": "1",
+    "hitCount": 0,
+    "missCount": 0,
+    "revalidationCount": 0,
+    "staleEntryCount": 0
+  },
+  "warnings": []
+}
+```
+
+Additional valid examples are under `test/fixtures/knowledge-manifest/`.
+
+## Limitations
+
+This contract supports public npm and PyPI package facts only. It does not implement networking, registry adapters, Knowledge Store persistence, GitHub fetching, private registries, version comparison, changelog interpretation, breaking-change analysis, impact analysis, migration planning, AI/LLM reasoning, or a research CLI command.

package/docs/MVP-02-Knowledge-Research-Orchestration.md

@@ -0,0 +1,62 @@
+# MVP-02 Knowledge Research Orchestration
+
+MVP-02-07 connects the validated Research Plan, injected npm/PyPI registry adapters, and Source Provenance Resolution into one deterministic internal execution. It consumes the Project Manifest lineage already captured by Research Planning; it does not rescan a repository or read package-manager files.
+
+> Knowledge Research Orchestration executes and combines deterministic research components. It does not publish the Knowledge Manifest, compare versions, or perform AI reasoning.
+
+```text
+Loaded Project Manifest → Research Plan → bounded adapter research
+                                           │
+                                           ▼
+                              complete registry result set
+                                           │
+                                           ▼
+                                Source Provenance Resolution
+                                           │
+                                           ▼
+                           internal KnowledgeResearchResult
+```
+
+## Input, adapter selection, and execution
+
+The orchestrator first calls `validateResearchPlan`. An invalid plan is a fatal internal contract failure and no adapter is invoked. A researchable package selects its adapter solely through `package.registry`: `npm` selects the npm-compatible adapter and `pypi` selects the PyPI adapter. Package-manager, installer, manifest, dependency type, and package-name heuristics never affect selection.
+
+Adapters are injected explicitly. The default concurrency is four and the accepted range is 1–32. A small worker pool schedules at most that many requests at once; output is sorted after all work completes, so adapter completion order does not affect the result. Cancellation is intentionally deferred because current adapters do not expose an orchestration-level signal contract.
+
+Registry base URLs and TTLs remain adapter configuration rather than a duplicate orchestration policy. The orchestration policy presently owns only bounded concurrency; it accepts no credentials, private-registry settings, installer configuration, or source-crawling policy.
+
+Each package is isolated. Adapter `resolved`, `partial`, `notFound`, and `unavailable` facts are preserved. A missing adapter or unexpected adapter exception becomes a sanitized `unavailable` result with `REGISTRY_UNAVAILABLE`; no stack, response, header, token, or cache path is retained. A missing adapter is a defensive integration diagnostic, not a new public warning code.
+
+## Result contract
+
+`KnowledgeResearchResult` is internal and starts at `resultVersion: "1"`; it is independent from the Knowledge Manifest schema version. It contains:
+
+- immutable Project Manifest lineage and Research Plan version;
+- execution timestamps, duration, concurrency, total and per-registry invocation counts, source/cache/warning counts, and a sorted per-package outcome list;
+- package records close to the Knowledge Manifest package shape, with all Research Plan occurrences attached;
+- normalized source records, warnings, invalid occurrences, and unsupported ecosystem aggregates;
+- canonical summary totals.
+
+Invalid references remain in `invalidOccurrences`, with an `INVALID_PACKAGE_REFERENCE` warning that has no invented package ID. Unsupported ecosystem aggregates remain in `unsupported` and do not invoke an adapter or create a fake npm/PyPI package. `partialFailureCount` counts only researched package records whose status is not `resolved`; invalid occurrences are reported separately. A registry-resolved package becomes `partial` when Source Provenance Resolution reports a `SOURCE_CONFLICT` for that package. Missing optional URLs do not make a package partial.
+
+## Sources, cache outcomes, and warnings
+
+Source Provenance Resolution runs once after every registry result is available. This allows source conflicts and future corroboration rules to see the complete package result set. Its package source IDs replace adapter-only package source IDs, and every source reference is validated before the result is returned.
+
+Cache aggregation preserves current adapter meanings:
+
+| Outcome | Counted as |
+| --- | --- |
+| `hit` | fresh cache hit |
+| `miss` | fetched after no usable cache entry |
+| `revalidated` | expired entry replaced by a normal fresh fetch, not HTTP `304` |
+| `corrupted-replaced` | corrupted entry replaced after a valid fetch |
+| `corrupted` | detected corruption without a successful replacement |
+
+Adapter, provenance, and invalid-reference warnings are normalized, deduplicated, and sorted by package ID, code, source ID, and message. Unsupported planning warnings remain internal planning metadata until a later public contract explicitly adopts them.
+
+## Invariants and privacy
+
+The runtime validator checks Research Plan lineage, one result per researchable package, status/cache/summary arithmetic, source and warning references, occurrence attachment, canonical ordering, provenance graph validity, timestamps, and result version. It rejects absolute local paths, credentials, query tokens, cache keys, store paths, ETag/Last-Modified values, raw headers, response bodies, and stack traces.
+
+No public artifact is written. Knowledge Manifest assembly, a research CLI, GitHub/documentation fetching, retry orchestration, version comparison, breaking-change analysis, and AI behavior remain outside MVP-02-07.

package/docs/MVP-02-Knowledge-Store.md

@@ -0,0 +1,87 @@
+# MVP-02 Lightweight Knowledge Store
+
+MVP-02-03 implements the smallest useful part of the Knowledge Store: a private, deterministic filesystem cache for future npm and PyPI registry adapters. It prevents repeat retrieval work once adapters exist, while keeping cache implementation details out of the portable Knowledge Manifest.
+
+The approved MVP-02 architecture remains the long-term direction. This implementation deliberately does not yet provide its complete content-addressed evidence store, HTTP validator handling, or multi-repository configuration. It is easy to delete and rebuild because it stores only cache envelopes, never a repository scan or public artifact.
+
+## Boundary and location
+
+The store is private to MVP-02 implementation:
+
+```text
+Research package
+        │
+        ▼
+Lightweight Knowledge Store
+        │
+        ▼
+Registry adapter decision
+```
+
+Its default root is relative to the process working directory:
+
+```text
+.upgradelens/cache/knowledge/v1/
+```
+
+Internal callers may configure a different root, which makes the store straightforward to test. The directory layout, filesystem path, cache key, and envelope are not part of either the Project Manifest or Knowledge Manifest contract. No downstream MVP consumes this directory.
+
+## Identity and key
+
+Each entry is package/source-centric. `createCacheIdentity` accepts exactly these portable fields:
+
+- `adapter` (for example, `npm` or `pypi`)
+- `resourceKind` (for example, `registry-package`)
+- `packageId` (for example, `npm:react` or `npm:@scope/package`)
+- `resourceVariant` (for example, `full-metadata`)
+- `adapterVersion`
+
+The cache calculates a SHA-256 filename from the canonical JSON identity. Scoped npm names and other package characters therefore never become directory names. Repository paths, Project Manifest digests, timestamps, retry counters, credentials, authorization, user agents, and volatile request headers are deliberately absent from this identity. The same resource request can be reused across runs without exposing its storage key in a public artifact.
+
+## Envelope contract
+
+Every stored JSON file has this strict internal shape:
+
+```json
+{
+  "envelopeVersion": "1",
+  "identity": {
+    "adapter": "npm",
+    "resourceKind": "registry-package",
+    "packageId": "npm:react",
+    "resourceVariant": "full-metadata",
+    "adapterVersion": "1"
+  },
+  "storedAt": "2026-07-14T00:00:00.000Z",
+  "expiresAt": "2026-07-15T00:00:00.000Z",
+  "bodyDigest": "sha256:<64 lowercase hexadecimal characters>",
+  "body": {}
+}
+```
+
+`bodyDigest` is SHA-256 over UTF-8 canonical JSON for `body`: object keys are sorted recursively and array order is preserved. Functions, symbols, BigInt, undefined values, non-finite numbers, circular references, and non-plain objects are rejected. This makes logically equal JSON objects produce the same digest regardless of insertion order, while preserving meaningful list order.
+
+The store rejects envelope fields beyond this contract and validates the version, identity, dates, expiry order, body digest, and body privacy on every read. Envelopes never persist cache keys, absolute local paths, credentials, authorization data, HTTP headers, ETag, Last-Modified, or raw error stacks.
+
+## Read states and TTL
+
+`cache.read(identity)` returns a structured result instead of turning normal cache conditions into exceptions:
+
+| State | Meaning |
+| --- | --- |
+| `fresh` | A valid entry exists and `now < expiresAt`; its body and metadata are returned. |
+| `expired` | A valid entry exists and `now >= expiresAt`; its body and metadata are returned for the adapter to decide whether stale data is usable. |
+| `missing` | No entry exists. |
+| `corrupted` | The file cannot be trusted (for example invalid JSON, wrong identity, unsupported envelope version, invalid dates, or digest mismatch). The body is never returned. |
+
+TTL is caller policy. `cache.write(identity, body, { ttlMs })` requires a non-negative integer TTL and computes `expiresAt = storedAt + ttlMs` using an injectable clock. The cache does not select registry TTLs, refresh records, make stale-fallback decisions, schedule invalidation, cache failures, or clean old files in the background.
+
+Operational filesystem failures—such as permission errors, unavailable storage, or an atomic rename failure—throw focused internal `KnowledgeCacheError` values with stable codes. They are not converted into registry warnings by this layer.
+
+## Atomic writes and concurrency
+
+Writes create parent directories, serialize the full envelope, write and flush a temporary file beside the target, close it, then rename it to the SHA-256 final filename. A failure cannot leave a partial final entry; temporary-file cleanup is best effort. The implementation intentionally does not provide locks or cross-process coordination. Concurrent-write policy is deferred until registry orchestration has a concrete need.
+
+## Intentional limitations
+
+MVP-02-03 does not implement HTTP fetching, npm/PyPI adapters, ETag, Last-Modified, conditional requests, shared-cache configuration, databases, background cleanup, distributed locking, portable evidence bundles, Knowledge Manifest generation, version analysis, or AI behavior. Those capabilities remain outside this small internal cache layer.

package/docs/MVP-02-PyPI-Registry-Adapter.md

@@ -0,0 +1,97 @@
+# MVP-02 PyPI Registry Adapter
+
+MVP-02-05 adds an internal adapter for public Python package facts from PyPI. It consumes a package-centric Research Plan record, uses the private Knowledge Store, and returns internal package, registry-source, cache-outcome, and warning records. It does not assemble or write a Knowledge Manifest.
+
+> The `pypi:` identity represents the PyPI registry ecosystem. It does not mean that the source project uses pip.
+
+## Python installers and PyPI identity
+
+pip, pip-tools, Poetry, uv, and Pipenv can all declare the same public PyPI project. The adapter uses only the normalized registry identity:
+
+```text
+pypi:<normalized-name>
+```
+
+Names follow the PyPA-compatible rule already used by Research Planning: lowercase and collapse runs of `.`, `_`, and `-` to `-`. For example, `LangChain_OpenAI` and `langchain-openai` identify `pypi:langchain-openai`; `zope.interface` identifies `pypi:zope-interface`.
+
+The adapter does not invoke installer CLIs and does not read `pip.conf`, Poetry/uv/Pipenv lockfiles, requirements options, or repository configuration. Installer identity never enters requests or cache keys.
+
+## Public API and request construction
+
+The primary official API is PyPI project JSON:
+
+```text
+GET https://pypi.org/pypi/<normalized-project-name>/json
+```
+
+The default metadata base is `https://pypi.org`; the future JSON Index base is `https://pypi.org/simple`. Both may be supplied as explicit HTTPS policy inputs for tests or later orchestration. HTTP, credentials, query-bearing bases, and repository-derived custom indexes are rejected.
+
+MVP-02-05 makes only the project JSON request. It does not scrape HTML, search PyPI, or issue a second Index API request. The existing bounded JSON fetcher provides GET-only requests, JSON `Accept`, a stable UpgradeLens user agent, omitted credentials, disabled redirects, timeout, and response-size limits.
+
+PyPI project JSON is generally smaller than a full npm packument, but established projects can still exceed 1 MB. The adapter owns an **8 MiB** default (`8 * 1024 * 1024`). Callers may explicitly override `maxResponseBytes` for tests or a future execution policy; it must be a positive safe integer and must not exceed the 64 MiB safety ceiling. This execution setting is not emitted in the Knowledge Manifest. Oversized project JSON retains the internal `PYPI_RESPONSE_TOO_LARGE` code, maps to the existing public `REGISTRY_RESPONSE_INVALID` warning, and is never cached.
+
+## Cache behavior
+
+The private [Lightweight Knowledge Store](MVP-02-Knowledge-Store.md) uses this identity:
+
+```json
+{
+  "adapter": "pypi",
+  "resourceKind": "registry-package",
+  "packageId": "pypi:fastapi",
+  "resourceVariant": "project-json",
+  "adapterVersion": "1"
+}
+```
+
+The default TTL is 24 hours, supplied by the adapter rather than the generic store. Fresh entries are normalized without a request. Missing entries fetch, validate, atomically cache, and normalize. Expired entries fetch a replacement; failure produces unavailable facts rather than silently returning stale data. Corrupted entries are never used and are overwritten only after a valid response. Cache outcomes do not expose cache paths, filenames, keys, validators, or envelopes.
+
+## Project JSON normalization
+
+The adapter validates `info`, `info.name`, and the requested normalized identity before caching. `info.name` can differ in case, underscore, dot, or hyphen spelling only when its normalized form matches the requested `pypi:` identity. `info.version` is optional: if absent or malformed, the adapter does not select another release and returns partial facts.
+
+`info.version` is the only registry-designated latest fact. It is recorded as `selection: project-info-version`; no PEP 440 parsing, semantic ordering, prerelease filtering, range comparison, or upgrade recommendation occurs.
+
+When project JSON exposes `releases`, release identifiers are sorted with code-unit lexical ordering. Each entry contains only the opaque version, earliest valid upload time, PyPI version page, nullable yanked state, and source ID. File URLs, hashes, wheels, dependency graphs, and build details are omitted. The normalizer is isolated so a later task can use the JSON Index API without changing common output fields.
+
+Yanked state uses only explicit file metadata:
+
+- all files explicitly `yanked: true` → `true`;
+- at least one file explicitly `yanked: false` → `false`;
+- otherwise → `null`.
+
+Missing file metadata is never converted to `false`.
+
+## Metadata and Project URLs
+
+The adapter normalizes a safe summary/description, preferring `license_expression` over `license`, and retains an explicit Development Status classifier in the existing `projectStatus` field. It does not infer licenses from classifiers or assign risk/recommendation meaning to project status.
+
+Publisher-controlled `info.project_urls` labels are classified case-insensitively:
+
+- `Documentation` / `Docs` → `documentationUrl`
+- `Source` / `Repository` / `Code` → `repositoryUrl`
+- `Issues` / `Tracker` / `Bug Tracker` → `issueUrl`
+- `Homepage` / `Home` → `homepageUrl`
+
+`Changelog`, `Changes`, `Release Notes`, and `Releases` are retained only as sanitized internal source candidates for Source Provenance Resolution; they are not public metadata fields and are not emitted in a Knowledge Manifest until later assembly. Unknown labels are ignored rather than guessed from domains or URL paths. `home_page` is used only as a safe homepage fallback when no classified homepage exists.
+
+Only HTTPS public URLs are retained. Credentialed URLs, query strings, and fragments are rejected or removed. `git+https:` repository URLs are normalized to HTTPS and GitHub `.git` suffixes are removed for display. Documentation and repository links remain publisher-trust candidates; MVP-02-05 does not corroborate them as official sources.
+
+## Source provenance and errors
+
+A successful result creates a registry source such as `pypi:fastapi:registry`, with registry authority, publisher trust, PyPI project/API URLs, supported fact roles, portable content digest, retrieval timestamp, and fresh snapshot. Store internals, ETag, Last-Modified, headers, credentials, and local paths never appear in this output.
+
+| Condition | Package status | Warning |
+| --- | --- | --- |
+| PyPI `404` | `notFound` | `PACKAGE_NOT_FOUND` |
+| PyPI `429` | `unavailable` | `REGISTRY_RATE_LIMITED` |
+| `5xx`, timeout, or transport failure | `unavailable` | `REGISTRY_UNAVAILABLE` |
+| Invalid JSON/media/size/project shape/name mismatch | `unavailable` | `REGISTRY_RESPONSE_INVALID` |
+
+Errors are sanitized; raw response bodies, headers, stack traces, tokens, credentialed URLs, and cache details are never returned. Negative responses, invalid responses, rate limits, and transport failures are not cached.
+
+All response bodies are consumed or explicitly cancelled. In particular, unused non-200 responses and wrong-media responses are cancelled, and an oversized stream cancels its reader before releasing the lock. Timeout handles are cleared in `finally` after every outcome. Online CLI research injects a scoped, CLI-owned fetch and closes its Agent after completion; direct callers retain ownership of injected fetch resources.
+
+## Intentional limitations
+
+This adapter does not implement private indexes, authentication, Python installer execution/configuration/lockfile parsing, dependency resolution, PEP 440 comparison, target selection, documentation fetching, GitHub Releases, source corroboration, PyPI JSON Index requests, Knowledge Manifest orchestration, a research CLI, breaking-change analysis, or AI behavior.