@thomasminh1995/depverdict 0.6.0-alpha.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (252) hide show
  1. package/LICENSE +21 -0
  2. package/README.md +518 -0
  3. package/bin/depverdict.js +7 -0
  4. package/bin/upgradelens.js +7 -0
  5. package/docs/GR-01-Semantic-Grounding-Failure-Analysis.md +309 -0
  6. package/docs/GR-02-Versioned-Action-Evaluation-Criteria.md +187 -0
  7. package/docs/GR-03-Extractive-Contract-Safety-Experiment.md +242 -0
  8. package/docs/GR-04-Versioned-Production-Extractive-Contract.md +154 -0
  9. package/docs/IA-01-Repository-Usage-Discovery.md +122 -0
  10. package/docs/IA-02-Repository-Impact-Analysis.md +118 -0
  11. package/docs/IA-03-Repository-Impact-Evidence.md +160 -0
  12. package/docs/IA-04-CLI-Orchestration.md +235 -0
  13. package/docs/IA-05-Real-Provider-Validation.md +235 -0
  14. package/docs/IA-05-VinGrade-Validation.md +339 -0
  15. package/docs/MVP-01.md +48 -0
  16. package/docs/MVP-02-Architecture.md +536 -0
  17. package/docs/MVP-02-CLI-HTTP-Runtime.md +50 -0
  18. package/docs/MVP-02-HTTP-Lifecycle.md +37 -0
  19. package/docs/MVP-02-Knowledge-Manifest-Generation.md +75 -0
  20. package/docs/MVP-02-Knowledge-Manifest.md +279 -0
  21. package/docs/MVP-02-Knowledge-Research-Orchestration.md +62 -0
  22. package/docs/MVP-02-Knowledge-Store.md +87 -0
  23. package/docs/MVP-02-PyPI-Registry-Adapter.md +97 -0
  24. package/docs/MVP-02-Research-Planning.md +196 -0
  25. package/docs/MVP-02-Source-Provenance.md +81 -0
  26. package/docs/MVP-02-npm-Registry-Adapter.md +99 -0
  27. package/docs/RR-01-End-to-End-and-Real-Provider-Validation.md +393 -0
  28. package/docs/RR-01-RERUN-Extractive-Contract-Validation.md +566 -0
  29. package/docs/RR-02-Full-Product-Workflow-and-Developer-CLI-UX-Review.md +35 -0
  30. package/docs/RR02-FIX-01-Persistent-Qualification-Resolution.md +32 -0
  31. package/docs/RR02-FIX-02-Stage-aware-CLI-Progress-and-Heartbeat.md +28 -0
  32. package/docs/RR02-FIX-03-npm-Capture-Evidence-Exclusion.md +33 -0
  33. package/docs/RR02-FIX-03A-Complete-Package-Exclusion-and-Evidence-Commit.md +223 -0
  34. package/docs/RR02-FIX-04-Event-loop-safe-Heartbeat.md +393 -0
  35. package/docs/RR02-FIX-05-Materialize-Persisted-Qualification.md +207 -0
  36. package/docs/RR02-RERUN-CLI-Qualification-Progress-UX-and-Package-Validation.md +342 -0
  37. package/docs/VinGrade-MVP-02-Validation.md +480 -0
  38. package/docs/VinGrade-RC02-Live-Validation.md +273 -0
  39. package/docs/ai-capability-discovery.md +573 -0
  40. package/docs/ai-engineering-production-readiness.md +390 -0
  41. package/docs/ai-engineering-review.md +251 -0
  42. package/docs/ai-runtime-governance-discovery.md +754 -0
  43. package/docs/architecture-overview.md +78 -0
  44. package/docs/cli-progress.md +111 -0
  45. package/docs/decisions/diff-01-brand-distribution-identity.md +450 -0
  46. package/docs/decisions/diff-02-identity-compatibility-contract.md +274 -0
  47. package/docs/decisions/diff-03-repository-docs-community-migration.md +165 -0
  48. package/docs/decisions/diff-04-release-evidence-gap-acceptance.md +110 -0
  49. package/docs/discovery/mvp-05-ai-migration-planning-discovery.md +350 -0
  50. package/docs/gateway-runtime-discovery.md +614 -0
  51. package/docs/live-ai-validation.md +299 -0
  52. package/docs/migration-planning-qualification-resolution.md +99 -0
  53. package/docs/migrations/upgradelens-to-depverdict.md +97 -0
  54. package/docs/mp-r03-deterministic-upgrade-decision-architecture.md +135 -0
  55. package/docs/mp-r04-evidence-bounded-migration-handoff-architecture.md +116 -0
  56. package/docs/mp-r05-product-completion-and-decision-first-cli-architecture.md +162 -0
  57. package/docs/mvp-05-deterministic-context-runtime.md +127 -0
  58. package/docs/mvp-05-migration-checklist-contract.md +166 -0
  59. package/docs/mvp-05-migration-checklist-orchestration.md +105 -0
  60. package/docs/mvp-05-migration-evaluation-and-qualification.md +135 -0
  61. package/docs/mvp-05-provider-neutral-generator.md +79 -0
  62. package/docs/ollama-local-smoke-validation.md +156 -0
  63. package/docs/openai-compatible-runtime-discovery.md +789 -0
  64. package/docs/openrouter-one-dependency-validation.md +205 -0
  65. package/docs/oss-02-package-guard-hardening-architecture.md +131 -0
  66. package/docs/oss-04-public-ci-package-metadata-architecture.md +142 -0
  67. package/docs/package-content-policy.md +98 -0
  68. package/docs/releases/v0.5.0-technical-preview.md +114 -0
  69. package/docs/releases/v0.6.0-alpha.1-depverdict-preview.md +101 -0
  70. package/docs/reviews/diff-02-identity-contract-compatibility.md +413 -0
  71. package/docs/reviews/diff-03-repository-docs-community-migration.md +364 -0
  72. package/docs/reviews/diff-04-depverdict-distribution-identity-readiness-rereview.md +453 -0
  73. package/docs/reviews/diff-04-fix-post-rename-identity-release-remediation.md +310 -0
  74. package/docs/reviews/diff-05-final-preview-distribution-qualification.md +561 -0
  75. package/docs/reviews/mvp-05-final-product-value-workflow-rereview.md +471 -0
  76. package/docs/reviews/mvp-05-product-workflow-review.md +605 -0
  77. package/docs/reviews/oss-01-duplicate-artifact-investigation-cleanup.md +411 -0
  78. package/docs/reviews/oss-02-package-guard-hardening.md +303 -0
  79. package/docs/reviews/oss-03-community-scaffolding.md +378 -0
  80. package/docs/reviews/oss-04-public-ci-package-metadata.md +439 -0
  81. package/docs/reviews/oss-05-technical-preview-qualification.md +482 -0
  82. package/docs/reviews/upgradelens-vs-upgradedepdetective-source-comparison.md +355 -0
  83. package/docs/reviews/v0.5.0-pre-release-smoke.md +263 -0
  84. package/docs/reviews/v0.5.0-version-bump-release-verification.md +306 -0
  85. package/docs/runtime-contract-discovery.md +521 -0
  86. package/docs/structured-output-compatibility-report.md +100 -0
  87. package/docs/ts-fix-01-exact-duplicate-occurrence-target-selection-architecture.md +111 -0
  88. package/docs/version-analysis-architecture.md +827 -0
  89. package/eval/README.md +86 -0
  90. package/eval/datasets/generic/declared-constraint.json +59 -0
  91. package/eval/datasets/generic/evidence-conflict.json +73 -0
  92. package/eval/datasets/generic/major-breaking-release.json +66 -0
  93. package/eval/datasets/generic/missing-evidence.json +46 -0
  94. package/eval/datasets/generic/patch-release-low.json +59 -0
  95. package/eval/datasets/node/axios-patch-low.json +59 -0
  96. package/eval/datasets/node/react-major-breaking.json +66 -0
  97. package/eval/datasets/node/react-minor-compatibility.json +66 -0
  98. package/eval/datasets/python/fastapi-deprecation.json +66 -0
  99. package/eval/datasets/python/pydantic-major-breaking.json +66 -0
  100. package/eval/migration-planning/golden-dataset-v2.json +306 -0
  101. package/eval/migration-planning/golden-dataset.json +214 -0
  102. package/eval/schemas/expected-result.schema.json +88 -0
  103. package/eval/schemas/golden-case.schema.json +181 -0
  104. package/package.json +57 -0
  105. package/schemas/ai-scorecard.schema.json +132 -0
  106. package/schemas/benchmark-report.schema.json +189 -0
  107. package/schemas/benchmark.schema.json +55 -0
  108. package/schemas/capability-profile.schema.json +44 -0
  109. package/schemas/conformance-report.schema.json +150 -0
  110. package/schemas/deployment-profile.schema.json +64 -0
  111. package/schemas/evaluation-report.schema.json +158 -0
  112. package/schemas/knowledge-evidence-bundle.schema.json +150 -0
  113. package/schemas/knowledge-manifest.schema.json +548 -0
  114. package/schemas/metrics.schema.json +178 -0
  115. package/schemas/migration-checklist-extractive-candidate.schema.json +42 -0
  116. package/schemas/migration-checklist.schema.json +706 -0
  117. package/schemas/migration-evaluation-dataset-v2.schema.json +208 -0
  118. package/schemas/migration-evaluation-dataset.schema.json +204 -0
  119. package/schemas/migration-planning-qualification-record.schema.json +234 -0
  120. package/schemas/project-manifest.schema.json +308 -0
  121. package/schemas/qualification-record.schema.json +56 -0
  122. package/schemas/repository-impact-evidence.schema.json +232 -0
  123. package/schemas/repository-impact.schema.json +202 -0
  124. package/schemas/upgrade-decision.schema.json +273 -0
  125. package/schemas/usage-index.schema.json +179 -0
  126. package/schemas/version-analysis.schema.json +449 -0
  127. package/src/ai-runtime-debug.js +325 -0
  128. package/src/ai-runtime-error.js +42 -0
  129. package/src/ai-runtime.js +174 -0
  130. package/src/ai-scorecard.js +204 -0
  131. package/src/ai-version-analysis.js +484 -0
  132. package/src/artifact-root-compatibility.js +91 -0
  133. package/src/benchmark-report.js +111 -0
  134. package/src/benchmark-runner.js +191 -0
  135. package/src/canonical-json.js +69 -0
  136. package/src/cli.js +1299 -0
  137. package/src/conformance-report.js +158 -0
  138. package/src/conformance-runner.js +253 -0
  139. package/src/constants.js +73 -0
  140. package/src/cooperative-scheduler.js +79 -0
  141. package/src/dependencies.js +44 -0
  142. package/src/dependency-ai-context.js +625 -0
  143. package/src/detectors.js +253 -0
  144. package/src/discovery.js +234 -0
  145. package/src/ecosystem-version-adapter.js +294 -0
  146. package/src/environment-compatibility.js +77 -0
  147. package/src/evaluation-comparator.js +158 -0
  148. package/src/evaluation-report.js +76 -0
  149. package/src/evaluation-runner.js +248 -0
  150. package/src/evidence-source-adapter.js +472 -0
  151. package/src/files.js +89 -0
  152. package/src/governance-diagnostics.js +64 -0
  153. package/src/governance-loader.js +63 -0
  154. package/src/governance-metadata.js +346 -0
  155. package/src/governance-validator.js +360 -0
  156. package/src/http/bounded-fetch.js +278 -0
  157. package/src/http/cli-http-runtime.js +44 -0
  158. package/src/impact/input-loader.js +157 -0
  159. package/src/impact/matcher.js +40 -0
  160. package/src/impact/repository-impact.js +199 -0
  161. package/src/impact/runtime.js +24 -0
  162. package/src/impact/status.js +62 -0
  163. package/src/impact/writer.js +30 -0
  164. package/src/impact-evidence/input-loader.js +202 -0
  165. package/src/impact-evidence/repository-impact-evidence.js +234 -0
  166. package/src/impact-evidence/runtime.js +16 -0
  167. package/src/impact-evidence/writer.js +30 -0
  168. package/src/index.js +563 -0
  169. package/src/installed-version-baseline.js +196 -0
  170. package/src/knowledge-cache.js +324 -0
  171. package/src/knowledge-evidence-bundle.js +101 -0
  172. package/src/knowledge-evidence-producer.js +233 -0
  173. package/src/knowledge-manifest-builder.js +188 -0
  174. package/src/knowledge-manifest-writer.js +32 -0
  175. package/src/knowledge-manifest.js +255 -0
  176. package/src/knowledge-research.js +615 -0
  177. package/src/metrics-engine.js +205 -0
  178. package/src/migration-checklist/ai-candidate.js +320 -0
  179. package/src/migration-checklist/assembler.js +37 -0
  180. package/src/migration-checklist/context-runtime.js +828 -0
  181. package/src/migration-checklist/evaluation/action-criteria.js +244 -0
  182. package/src/migration-checklist/evaluation/comparator-v2.js +332 -0
  183. package/src/migration-checklist/evaluation/comparator.js +279 -0
  184. package/src/migration-checklist/evaluation/dataset-v2.js +227 -0
  185. package/src/migration-checklist/evaluation/dataset.js +336 -0
  186. package/src/migration-checklist/evaluation/extractive-fixtures-v2.js +148 -0
  187. package/src/migration-checklist/evaluation/metrics-v2.js +226 -0
  188. package/src/migration-checklist/evaluation/metrics.js +158 -0
  189. package/src/migration-checklist/evaluation/qualification-v2.js +321 -0
  190. package/src/migration-checklist/evaluation/qualification.js +239 -0
  191. package/src/migration-checklist/evaluation/runner-v2.js +294 -0
  192. package/src/migration-checklist/evaluation/runner.js +194 -0
  193. package/src/migration-checklist/evaluation/scorecard-v2.js +106 -0
  194. package/src/migration-checklist/evaluation/scorecard.js +86 -0
  195. package/src/migration-checklist/extractive-candidate.js +166 -0
  196. package/src/migration-checklist/extractive-prompt.js +62 -0
  197. package/src/migration-checklist/generator.js +702 -0
  198. package/src/migration-checklist/grounding-policy.js +117 -0
  199. package/src/migration-checklist/input-loader.js +613 -0
  200. package/src/migration-checklist/migration-checklist.js +635 -0
  201. package/src/migration-checklist/presentation.js +292 -0
  202. package/src/migration-checklist/progress.js +96 -0
  203. package/src/migration-checklist/prompt.js +83 -0
  204. package/src/migration-checklist/qualification-guard.js +462 -0
  205. package/src/migration-checklist/qualification-resolution.js +122 -0
  206. package/src/migration-checklist/qualification-store.js +225 -0
  207. package/src/migration-checklist/runtime.js +205 -0
  208. package/src/migration-checklist/verification.js +134 -0
  209. package/src/migration-checklist/writer.js +35 -0
  210. package/src/openai-compatible-provider.js +451 -0
  211. package/src/orchestration/failure-log.js +32 -0
  212. package/src/orchestration/pipeline.js +200 -0
  213. package/src/orchestration/progress-events.js +337 -0
  214. package/src/orchestration/progress-reporter.js +131 -0
  215. package/src/orchestration/text-writer.js +22 -0
  216. package/src/portable.js +13 -0
  217. package/src/product-completion.js +249 -0
  218. package/src/project-manifest-input.js +90 -0
  219. package/src/project-manifest.js +141 -0
  220. package/src/python-requirements.js +137 -0
  221. package/src/registry/npm-packument.js +256 -0
  222. package/src/registry/npm-registry-adapter.js +262 -0
  223. package/src/registry/pypi-project.js +300 -0
  224. package/src/registry/pypi-registry-adapter.js +235 -0
  225. package/src/registry/sanitize-registry-body.js +51 -0
  226. package/src/renderers/console.js +160 -0
  227. package/src/renderers/impact-presentation.js +278 -0
  228. package/src/renderers/markdown.js +172 -0
  229. package/src/research-plan.js +455 -0
  230. package/src/runtime-conformance.js +275 -0
  231. package/src/source-provenance.js +393 -0
  232. package/src/source-url.js +62 -0
  233. package/src/structured-output-schema.js +66 -0
  234. package/src/target-selector.js +306 -0
  235. package/src/upgrade-decision/input-loader.js +107 -0
  236. package/src/upgrade-decision/presentation.js +43 -0
  237. package/src/upgrade-decision/runtime.js +21 -0
  238. package/src/upgrade-decision/upgrade-decision.js +626 -0
  239. package/src/upgrade-decision/writer.js +30 -0
  240. package/src/usage/analyzer-registry.js +63 -0
  241. package/src/usage/coverage.js +116 -0
  242. package/src/usage/input-loader.js +139 -0
  243. package/src/usage/js/analyzer.js +240 -0
  244. package/src/usage/js/parser.js +21 -0
  245. package/src/usage/runtime.js +187 -0
  246. package/src/usage/scope.js +44 -0
  247. package/src/usage/source-files.js +50 -0
  248. package/src/usage/usage-index.js +217 -0
  249. package/src/usage/writer.js +31 -0
  250. package/src/version-analysis-loader.js +203 -0
  251. package/src/version-analysis-manifest.js +314 -0
  252. package/src/version-analysis-writer.js +30 -0
@@ -0,0 +1,605 @@
1
+ # UpgradeLens MVP-05 — Full Product Workflow and Feature Alignment Review
2
+
3
+ ## 1. Executive Verdict
4
+
5
+ **`PARTIAL_MATCH`**
6
+
7
+ The UpgradeLens pipeline is operational, deterministic, well validated, and
8
+ safe against unsupported AI actions. It can discover a representative
9
+ polyglot repository, research public evidence, run provider-backed Version
10
+ Analysis, index JavaScript/TypeScript usage, preserve artifact lineage, and
11
+ produce a human-review-gated report.
12
+
13
+ It does not yet deliver the complete product outcome claimed for MVP-05. The
14
+ real VinGrade workflow produced no explicit upgrade recommendation, no
15
+ grounded migration action, no candidate location, no ordered roadmap, and no
16
+ verification or recovery plan. A Coding Agent had to inspect lockfiles, source,
17
+ tests, and runtime configuration to derive even “already at target” and “hold
18
+ the upgrade” conclusions.
19
+
20
+ | Decision | Result |
21
+ | --- | --- |
22
+ | v0.5.0 release readiness | **`NO_GO`** for the stated full dependency-upgrade intelligence and migration-roadmap positioning |
23
+ | Migration Checklist readiness | **`KEEP_EXPERIMENTAL`**; not ready for default enablement |
24
+ | Real-provider qualification | **`QUALIFIED`** for `openai-compatible / openai/gpt-5.5 / openai-compatible` |
25
+ | Product workflow readiness | Pipeline-ready; product outcome **partially ready** |
26
+ | Coding Agent handoff readiness | **Not ready for implementation**; useful only to start investigation |
27
+
28
+ This recommendation is based on product-output gaps reproduced on a real
29
+ repository, not on automated test failures.
30
+
31
+ ## 2. Environment and Scope
32
+
33
+ | Item | Value |
34
+ | --- | --- |
35
+ | UpgradeLens branch | `feat/mvp-05-evidence-migration-checklist` |
36
+ | UpgradeLens commit | `44eb942e21ea78550700990a00bd74635ce48bcc` (`fix: event-loop starvation`) |
37
+ | Package | `upgradelens@0.4.0` |
38
+ | Node.js | `v26.0.0` |
39
+ | npm | `11.12.1` |
40
+ | Platform | macOS arm64 |
41
+ | Representative repository | VinGrade |
42
+ | VinGrade commit | `25811ef997fcf45810105e89e4500688f28f7ba5` |
43
+ | VinGrade execution target | Clean isolated `git archive` snapshot |
44
+ | Provider/model | `openai-compatible` / `openai/gpt-5.5` |
45
+ | Provider debug output | Disabled |
46
+ | Qualification | Persisted, exact identity, `QUALIFIED` |
47
+
48
+ The original VinGrade checkout had a deleted tracked `Makefile` and unrelated
49
+ untracked user files. It was not modified. All workflow writes went to
50
+ isolated temporary snapshots.
51
+
52
+ The review covered the current source, schemas, tests, fixtures, public CLI,
53
+ packaged CLI, full online and offline workflows, failure handling, occurrence
54
+ identity, provider qualification, console/Markdown presentation, and Coding
55
+ Agent handoff.
56
+
57
+ ### Validation status
58
+
59
+ | Validation | Result |
60
+ | --- | --- |
61
+ | Canonical `npm run check` | 545 passed, 0 failed, 1 skipped |
62
+ | Skip | Local-loopback lifecycle test: sandbox does not expose listeners to that test |
63
+ | Package guard | 209 files after this report, 0 CLI-capture evidence, 15/15 required assets |
64
+ | Static lint | Blocked: repository defines no lint script |
65
+ | Schema validation | Passed through automated suite and runtime artifact writers/loaders |
66
+ | `git diff --check` | Passed |
67
+ | Fresh npm pack | Passed |
68
+ | Clean offline install | Passed using exact local dependency tarballs |
69
+ | Packaged version/help/import | Passed; version `0.4.0`, 408 public exports |
70
+ | Second-model comparison | Blocked: no second model configuration was available |
71
+ | Provider token/cost comparison | Blocked: production artifacts do not retain request usage/cost totals |
72
+
73
+ The canonical suite’s single skip is reported rather than counted as a pass.
74
+ No test, schema, fixture, timeout, policy, or source implementation was changed.
75
+
76
+ ## 3. Full Workflow Result
77
+
78
+ The main command was:
79
+
80
+ ```text
81
+ upgradelens analyze . --progress plain --experimental-migration-checklist
82
+ ```
83
+
84
+ The command ran from the isolated VinGrade snapshot. Public registry research
85
+ used fresh or cached evidence. Version Analysis and the eligible Migration
86
+ Checklist context used the configured real provider.
87
+
88
+ | Stage | Status | Input | Output | Duration | User-visible result |
89
+ | --- | --- | --- | --- | ---: | --- |
90
+ | Project Discovery | Passed | VinGrade source tree | `project-manifest.json` | 0.1s | 2 projects; Node and Python |
91
+ | Knowledge Research | Passed with warnings | 47 occurrences / 46 packages | `knowledge-manifest.json`, `knowledge-evidence-bundle.json` | 9.3s | Not shown in final summary |
92
+ | Version Analysis | Passed with partial results | Project, knowledge, 577 evidence records | `version-analysis.json` | 398.8s | Final summary showed 36 analyzed, 3 skipped, 8 failed |
93
+ | Repository Usage Discovery | Passed with warning | Source tree | `usage-index.json` | 0.2s | 17 dependencies, 152 symbols, 79 files; detail not shown in final summary |
94
+ | Repository Impact Analysis | Passed | Version Analysis and Usage Index | `repository-impact.json` | <0.1s | 0 impacted, 36 not impacted, 11 not analyzed |
95
+ | Repository Impact Evidence | Passed | Impact and lineage artifacts | `repository-impact-evidence.json` | <0.1s | 4 finding records, no affected files |
96
+ | Migration Checklist | Passed but incomplete | Full lineage and exact qualification | `migration-checklist.json` | 0.2s | 4 manual-review items, 0 grounded actions, 0 locations |
97
+ | Markdown Report | Passed | Presentation view models | `repository-impact.md` | <0.1s | Report path shown |
98
+
99
+ Total CLI time was 408.6 seconds; the capture wrapper measured 409.3 seconds.
100
+ All eight stages reached terminal success, but business completeness was
101
+ `INCOMPLETE`.
102
+
103
+ ### Artifact results
104
+
105
+ - Project Discovery: 2 projects, 47 dependency occurrences.
106
+ - Knowledge Research: 45 resolved packages, 1 unavailable package, 298
107
+ sources, 577 evidence records, 84 manifest warnings.
108
+ - Version Analysis: 36 analyzed, 3 skipped, 8 failed, 46 requiring review.
109
+ - Risk: 1 medium, 46 unknown.
110
+ - Usage: JavaScript/TypeScript coverage only; 1 parse warning.
111
+ - Impact: 4 breaking findings, none reported as impacted.
112
+ - Checklist: 47 dependency records, 4 finding records, 4 deterministic
113
+ manual-review items, no grounded action and no candidate location.
114
+
115
+ ### Provider requests
116
+
117
+ The one-call-per-eligible-context contract and retained result states show:
118
+
119
+ - 44 Version Analysis attempts: 36 successful and 8 HTTP 402
120
+ `INSUFFICIENT_CREDIT` failures;
121
+ - 3 skipped Version Analysis contexts made no request;
122
+ - 1 eligible Migration Checklist attempt failed and was contained as
123
+ `AI_RUNTIME_FAILED`;
124
+ - total real-provider attempts: **45**;
125
+ - hidden retries: **0**.
126
+
127
+ A separate loopback invalid-output fixture received exactly one simulated
128
+ request. It is not counted as a real-provider request.
129
+
130
+ ## 4. Core Feature Alignment Matrix
131
+
132
+ | Feature | Verdict | Evidence | Developer/team impact | v0.5.0 relevance |
133
+ | --- | --- | --- | --- | --- |
134
+ | Dependency Inventory | **MATCH** | Project Manifest preserved 47 occurrences across Node and Python with project, manifest, declaration, and dependency type | Team can enumerate direct declarations and distinguish occurrences | Required and ready |
135
+ | Upgrade Detection | **PARTIAL_MATCH** | Registry targets and missing-target states exist, but every resolvable declaration is analyzed; there is no separate “requires attention” selection | Teams receive broad analysis rather than a focused attention list | Required improvement |
136
+ | Upgrade Recommendation | **NOT_MATCH** | Version Analysis has status, risk, review reasons, and `nextAction`, but no `upgrade now / plan / keep / insufficient evidence` decision | A tech lead must infer whether any dependency should change | Release-critical |
137
+ | Reason and Evidence | **PARTIAL_MATCH** | Findings reference exact evidence IDs and target versions; output also clearly labels registry latest as a fact, not a recommendation | Good auditability, but reasons do not resolve the decision | Important |
138
+ | Repository Risk Assessment | **PARTIAL_MATCH** | JS/TS exact-symbol matching works; Python is not indexed, yet one LangSmith finding was reported `DEPENDENCY_NOT_USED` and `NOT_IMPACTED` despite real Python usage | False confidence is possible outside supported analyzer coverage | Release-critical |
139
+ | Dependency Classification | **NOT_MATCH** | Dependency type is recorded, but no framework/core, small package, internal package, or transitive classification exists | React and `remark-gfm` receive essentially the same decision treatment | Important |
140
+ | Prioritization | **NOT_MATCH** | No dependency or action priority, urgency, rank, or planning order is presented | Teams cannot turn the report into a work queue | Release-critical |
141
+ | Migration Roadmap | **NOT_MATCH** | Checklist had zero grounded actions/locations; the production prompt forbids prerequisites, order, rollback, and verification claims | The output is a review ledger, not a migration roadmap | Release-critical |
142
+ | Human Review | **MATCH** | 46/47 Version results required review; checklist always states human review is mandatory | Unsafe automation is prevented | Required and ready |
143
+ | Coding Agent Handoff | **PARTIAL_MATCH** | Stable IDs, evidence refs, findings, and limitations help investigation; the Agent still had to recover versions, files, tests, and runtime constraints from source | Agent cannot safely begin implementation from the artifact | Release-critical |
144
+ | Model Transparency | **PARTIAL_MATCH** | Checklist console shows qualification and runtime tuple; Version Analysis and Checklist JSON do not persist provider/model/request identity | Saved artifacts cannot independently establish which runtime produced results | Important |
145
+ | Evidence Freshness | **MATCH** | Source records retain `retrievedAt`, freshness, publication dates, and cache results | Reviewer can distinguish fresh, cached, and unavailable evidence | Required and ready |
146
+ | Offline Readiness | **MATCH** | Empty-cache offline run safely produced 47 not-analyzed results; fresh-cache offline research reused 45 packages without network | Offline limits are representable without inventing latest facts | Ready with UX caveat |
147
+ | Continuous Tracking Readiness | **MATCH** | Canonical IDs, schema versions, artifact digests, research IDs, context IDs, and lineage are stable | Future runs can be compared without name-only matching | Ready |
148
+ | CLI-first Delivery | **PARTIAL_MATCH** | One `analyze` command runs the pipeline with stable progress and CI-friendly plain mode; important decisions and failure next steps remain buried in artifacts | Operationally usable, but not yet outcome-first | Important |
149
+
150
+ **Count: 5 MATCH, 6 PARTIAL_MATCH, 4 NOT_MATCH.**
151
+
152
+ ## 5. Three Core Questions Assessment
153
+
154
+ ### Question 1 — Should the dependency be upgraded?
155
+
156
+ **No clear answer is produced.**
157
+
158
+ React is the decisive example:
159
+
160
+ - declared constraint: `^19.2.6`;
161
+ - lockfile installed version: `19.2.7`;
162
+ - reported target: `19.2.7`;
163
+ - Version Analysis current version: `null`;
164
+ - risk: `unknown`;
165
+ - next action: review before impact analysis.
166
+
167
+ The technically useful answer is “already at the target; do not perform a
168
+ version change.” UpgradeLens does not read the lockfile baseline and therefore
169
+ cannot produce that decision.
170
+
171
+ The small package `remark-gfm` is also locked at the reported `4.0.1` target,
172
+ but receives unknown risk and mandatory review instead of a concise “keep
173
+ current version” outcome.
174
+
175
+ ### Question 2 — Why, and what is the risk?
176
+
177
+ **Partially answered.**
178
+
179
+ Strengths:
180
+
181
+ - evidence references are exact and target-scoped;
182
+ - uncertainty and registry-latest limitations are explicit;
183
+ - findings are retained separately from deterministic impact matching;
184
+ - provider failure does not invent findings.
185
+
186
+ Weaknesses:
187
+
188
+ - the unresolved baseline prevents meaningful delta risk for most packages;
189
+ - 46/47 results remain `unknown`;
190
+ - no framework/core versus small-package context changes the presentation;
191
+ - Python usage is unsupported, but the report still says LangSmith is not
192
+ impacted because the dependency is not used.
193
+
194
+ A tech lead must inspect multiple JSON artifacts and repository source to
195
+ understand the actual risk.
196
+
197
+ ### Question 3 — What is the migration roadmap?
198
+
199
+ **Not answered.**
200
+
201
+ The checklist contains:
202
+
203
+ - 4 deterministic manual-review placeholders;
204
+ - 0 grounded actions;
205
+ - 0 AI-selected official guidance items;
206
+ - 0 candidate locations;
207
+ - no order, preconditions, verification strategy, rollout, or recovery step.
208
+
209
+ The extractive production contract intentionally prevents invented actions,
210
+ which is the correct safety behavior. It also means the current artifact is
211
+ not the migration roadmap described by the product positioning.
212
+
213
+ ### Planning-meeting usability
214
+
215
+ The report is useful as a safe evidence and uncertainty inventory. It is not
216
+ ready to drive an upgrade-planning meeting without additional lockfile,
217
+ source, and test investigation.
218
+
219
+ ## 6. Scenario Results
220
+
221
+ ### Scenario A — Representative real project
222
+
223
+ **Result: PARTIAL_MATCH**
224
+
225
+ VinGrade is a real polyglot project with 47 occurrences, 46 packages, a React
226
+ frontend, Python backend, core and small packages, and real source usage.
227
+
228
+ The main command correctly ran all stages in order and produced the complete
229
+ artifact chain. Critical loader/lineage failures stop their commands.
230
+ Package-local provider failures are retained without deleting successful
231
+ results.
232
+
233
+ The workflow still ends with no recommendation and no roadmap.
234
+
235
+ ### Scenario B — Core framework: React
236
+
237
+ **Result: NOT_MATCH**
238
+
239
+ React is already locked at the reported target `19.2.7`. UpgradeLens treats
240
+ the baseline as unresolved, assigns unknown risk, and produces a private
241
+ React Server Components path finding that does not match repository usage.
242
+ It does not say “already current,” propose no change, identify architecture
243
+ areas, or provide framework-level verification.
244
+
245
+ ### Scenario C — Small package: `remark-gfm`
246
+
247
+ **Result: NOT_MATCH**
248
+
249
+ `remark-gfm` is also already locked at its target `4.0.1`. The output lists
250
+ type/documentation changes, leaves risk unknown, and requires human review.
251
+ There is no concise keep-current conclusion. Dependency type is preserved,
252
+ but there is no small-package classification or proportional UX.
253
+
254
+ ### Scenario D — Insufficient evidence
255
+
256
+ **Result: MATCH for safety; PARTIAL_MATCH for UX**
257
+
258
+ With an empty offline cache:
259
+
260
+ - 46 packages were unavailable;
261
+ - evidence count was 0;
262
+ - all 47 Version Analysis results were skipped;
263
+ - no model was called;
264
+ - all 47 required review;
265
+ - Migration Checklist generated no grounded action.
266
+
267
+ No latest version, command, source location, or migration instruction was
268
+ invented. The final console output does not tell the user to populate the
269
+ cache or rerun research online; that next step remains inside artifacts.
270
+
271
+ ### Scenario E — Mixed occurrences
272
+
273
+ **Result: MATCH for identity; PARTIAL_MATCH for presentation**
274
+
275
+ The duplicate `langchain-openai` declarations remained separate:
276
+
277
+ - unversioned occurrence: distinct context/result ID, skipped with
278
+ `BASELINE_UNSUPPORTED`;
279
+ - `>=0.3.0` occurrence: distinct context/result ID, analyzed.
280
+
281
+ No occurrence was silently chosen or merged, and one skipped occurrence did
282
+ not change the other result. Markdown shows two identical
283
+ `langchain-openai` headings without the declaration/context identity, which is
284
+ ambiguous for a human reviewer.
285
+
286
+ ### Scenario F — Failure and recovery
287
+
288
+ | Failure | Exit | Result |
289
+ | --- | ---: | --- |
290
+ | Missing upstream Project Manifest | 1 | Correct stop; raw `ENOENT` path, no stage label or concrete next command |
291
+ | Missing AI runtime configuration | 1 | Clear configuration message; does not explain provider/model setup beyond endpoint |
292
+ | Invalid artifact lineage | 1 | Precise digest and repository-name mismatch; no artifact replacement |
293
+ | Invalid structured response | 0 | Fail-closed result artifact with `OUTPUT_JSON_INVALID`, but CLI prints “complete” and exits success |
294
+ | Real provider insufficient credit | 0 for full workflow | 8 package failures retained; final summary shows failure count, but credit recovery is not shown |
295
+ | Migration Checklist provider failure | 0 for full workflow | Contained as manual review; no unsupported action |
296
+ | Pipeline cancellation | 130 | Existing same-HEAD packaged capture proves controlled cancellation and no partial Usage Index |
297
+
298
+ The runtime performs no automatic retry. The invalid-output fixture received
299
+ one request.
300
+
301
+ ### Coding Agent handoff
302
+
303
+ **Result: NOT READY**
304
+
305
+ Using only the source repository and Migration Checklist, the Agent concluded:
306
+
307
+ - React, React DOM, and PDF.js were already at target;
308
+ - LangSmith should be held pending API evidence;
309
+ - Node runtime compatibility and real test prerequisites need review.
310
+
311
+ Those conclusions required repository re-research. The checklist itself
312
+ provided no concrete change, test, file, location, or target approval.
313
+
314
+ ### Cross-model readiness
315
+
316
+ **Execution blocked.** Only `openai-compatible / openai/gpt-5.5` was
317
+ configured. No Claude Sonnet configuration was available, so no cross-model
318
+ quality claim is made.
319
+
320
+ The qualification resolver distinguishes exact qualified, missing, mismatch,
321
+ corrupted, insufficient, and rejected records. It does not expose a separate
322
+ user-facing compatible/qualified/experimental/rejected/unknown model taxonomy
323
+ in saved Version Analysis artifacts.
324
+
325
+ ### Offline readiness
326
+
327
+ **Result: MATCH**
328
+
329
+ The empty-cache offline run safely represented external verification as
330
+ unavailable. A separate offline fresh-cache run reused 45 packages, retained 1
331
+ cache miss, performed no registry request, and preserved evidence freshness.
332
+
333
+ Project-only discovery and usage analysis continue to work offline. The final
334
+ console summary should state the offline evidence limitation more directly.
335
+
336
+ ## 7. CLI UX Review
337
+
338
+ ### What works
339
+
340
+ - `upgradelens analyze` is the correct single workflow command.
341
+ - Stage order and plain progress are stable and CI-friendly.
342
+ - Heartbeats make the long provider stage visibly active.
343
+ - Final summary clearly shows analyzed/skipped/failed/review counts.
344
+ - Qualification status, source, ID, runtime tuple, and experimental override
345
+ are visible when Migration Checklist runs.
346
+ - Secret-like provider values are not printed.
347
+ - Missing configuration and lineage failures are concise.
348
+ - Output paths are portable relative paths in the success summary.
349
+
350
+ ### What is difficult
351
+
352
+ - The summary leads with counts, not “upgrade / keep / investigate” decisions.
353
+ - `--help` describes commands and artifacts, not the three product outcomes.
354
+ - Provider setup is not explained beyond environment variable errors.
355
+ - Research warnings and unavailable-package identity are absent from final
356
+ output.
357
+ - `NOT_IMPACTED` appears without analyzer-coverage context.
358
+ - A package-local provider/output failure can exit 0 and print “complete.”
359
+ - Migration Checklist exposes internal qualification details before any useful
360
+ action.
361
+ - The developer must open multiple JSON artifacts to find next steps and
362
+ limitations.
363
+ - The full help page exposes evaluation, scorecard, benchmark, conformance,
364
+ and governance commands at the same level as the primary workflow.
365
+
366
+ ### Recommended default output
367
+
368
+ Lead with a dependency decision table:
369
+
370
+ ```text
371
+ Dependency Decision Current → Target Risk Why / next step
372
+ react Keep current 19.2.7 → 19.2.7 Low Already at target
373
+ axios Insufficient evidence 1.17.0 → 1.18.1 Unknown Review release delta
374
+ vite Not analyzed unknown Unknown Collect target evidence
375
+ ```
376
+
377
+ Then show affected areas, human approvals, and the report/checklist paths.
378
+ Keep qualification digests and detailed reason codes behind a verbose or
379
+ diagnostic view.
380
+
381
+ ### Capture references
382
+
383
+ Captures are stored locally under:
384
+
385
+ ```text
386
+ .upgradelens/review/mvp-05-workflow/
387
+ ```
388
+
389
+ Important entries:
390
+
391
+ - `003-vingrade-full-offline-qualified/`
392
+ - `007-vingrade-full-online-qualified/`
393
+ - `008-missing-ai-runtime-config/`
394
+ - `009-invalid-artifact-lineage/`
395
+ - `010-invalid-structured-response/`
396
+ - `013-packaged-version/`
397
+ - `014-packaged-help/`
398
+ - `016-agent-handoff-assessment.md`
399
+ - `017-offline-fresh-cache-research/`
400
+ - `018-post-report-npm-pack/`
401
+ - `019-final-package-guard/`
402
+
403
+ Each command entry retains command, working directory, safe runtime identity,
404
+ exit code, exact stdout/stderr, duration, and artifact inventory. Eight
405
+ important final screens were rendered from exact raw transcripts; they are
406
+ not presented as native terminal screenshots.
407
+
408
+ ## 8. Defects and Gaps
409
+
410
+ ### Blocker
411
+
412
+ #### B-01 — The workflow does not produce an upgrade decision or migration roadmap
413
+
414
+ - Reproduction: run the qualified online VinGrade workflow.
415
+ - Expected: explicit upgrade/plan/keep/insufficient decision plus a safe,
416
+ ordered roadmap for actionable dependencies.
417
+ - Actual: 47 dependencies, 4 breaking findings, 4 manual-review placeholders,
418
+ 0 grounded actions, 0 locations, and no roadmap.
419
+ - Product impact: Questions 1 and 3 are unanswered; the headline MVP-05
420
+ product value is not delivered.
421
+ - Evidence: capture `007`; `migration-checklist.json`; Coding Agent handoff.
422
+
423
+ ### High
424
+
425
+ #### H-01 — Lockfile baselines are ignored
426
+
427
+ - Reproduction: inspect React, React DOM, PDF.js, and `remark-gfm` in VinGrade.
428
+ - Expected: use the exact package-lock version as current baseline.
429
+ - Actual: current version is `null`; target versions already installed are
430
+ treated as uncertain registry facts.
431
+ - Product impact: unnecessary provider work, unknown risk, and no keep-current
432
+ decision.
433
+ - Evidence: `FE/package-lock.json` versus `version-analysis.json`.
434
+
435
+ #### H-02 — Unsupported Python usage can be presented as not impacted
436
+
437
+ - Reproduction: the LangSmith breaking finding in the online run.
438
+ - Expected: usage coverage unknown because no Python analyzer exists.
439
+ - Actual: `DEPENDENCY_NOT_USED`, `NOT_IMPACTED`; VinGrade imports and uses
440
+ `langsmith.Client`.
441
+ - Product impact: false repository-risk confidence.
442
+ - Evidence: `repository-impact-evidence.json`, VinGrade
443
+ `evaluation/langsmith/*.py`, capture `007`.
444
+
445
+ #### H-03 — Package-level provider/output failures can look successful
446
+
447
+ - Reproduction: invalid structured-response capture and the real provider
448
+ credit exhaustion.
449
+ - Expected: nonzero or explicit partial-success exit with failed package and
450
+ recovery action in CLI output.
451
+ - Actual: exit 0 and “AI Version Analysis complete”; failed result exists only
452
+ in JSON. The full run also exits 0 with 8 failed dependencies.
453
+ - Product impact: CI or a developer can treat incomplete analysis as success.
454
+ - Evidence: captures `007` and `010`.
455
+
456
+ ### Medium
457
+
458
+ #### M-01 — Runtime identity is not self-contained in analysis artifacts
459
+
460
+ - Reproduction: inspect the saved Version Analysis and Migration Checklist
461
+ artifacts from capture `007`.
462
+ - Expected: saved provider, model, runtime mode, and qualification status.
463
+ - Actual: console checklist presentation shows that identity, while the JSON
464
+ retains only UpgradeLens generator identity and lineage.
465
+ - Product impact: a later reviewer cannot independently establish which AI
466
+ runtime produced the saved result.
467
+ - Evidence: capture `007`; `version-analysis.json`;
468
+ `migration-checklist.json`.
469
+
470
+ #### M-02 — Failure recovery guidance is inconsistent
471
+
472
+ - Reproduction: run the missing-upstream, missing-configuration, and invalid-
473
+ lineage scenarios.
474
+ - Expected: stage name, plain-language cause, and a concrete safe next step.
475
+ - Actual: missing upstream input exposes a raw `ENOENT` path; missing
476
+ configuration names a variable; lineage failure gives no next command.
477
+ - Product impact: recovery depends on internal CLI knowledge.
478
+ - Evidence: captures `004`, `008`, and `009`.
479
+
480
+ #### M-03 — Duplicate occurrences are hard to distinguish in Markdown
481
+
482
+ - Reproduction: open the two `langchain-openai` sections in the VinGrade
483
+ Markdown report.
484
+ - Expected: occurrence labels that identify manifest, declaration, or context.
485
+ - Actual: JSON identity is correct, but duplicate headings omit declared
486
+ version, manifest occurrence, and context ID.
487
+ - Product impact: a reviewer can attach a result to the wrong declaration.
488
+ - Evidence: capture `007`; `repository-impact.md`;
489
+ `version-analysis.json`.
490
+
491
+ #### M-04 — The required review report is publishable by default
492
+
493
+ - Reproduction: run the post-report npm pack in capture `018`.
494
+ - Expected: internal review reports excluded unless they are intentional
495
+ public package documentation.
496
+ - Actual: the package includes `docs/**` except CLI-capture trees, so this
497
+ report is present in the 209-file tarball.
498
+ - Product impact: internal review conclusions can be published unintentionally.
499
+ - Evidence: captures `018` and `019`.
500
+
501
+ ### Low
502
+
503
+ #### L-01 — Snapshot directory name becomes the repository name
504
+
505
+ - Reproduction: run discovery from the isolated `vingrade-online` snapshot.
506
+ - Expected: repository identity derived from project metadata or clearly
507
+ labeled as the directory name.
508
+ - Actual: the displayed repository name is `vingrade-online`, not VinGrade.
509
+ - Product impact: report identity is noisy in temporary and CI workspaces.
510
+ - Evidence: captures `005` and `007`.
511
+
512
+ #### L-02 — Help is command-first rather than outcome-first
513
+
514
+ - Reproduction: run the packaged `--help` smoke test.
515
+ - Expected: the primary upgrade decision, risk, and roadmap outcomes are
516
+ explained before diagnostic commands.
517
+ - Actual: commands and artifact contracts dominate the help output.
518
+ - Product impact: a first-time user must infer the intended product workflow.
519
+ - Evidence: capture `014`.
520
+
521
+ #### L-03 — Final summary omits the best next action
522
+
523
+ - Reproduction: inspect the online and empty-cache offline final summaries.
524
+ - Expected: name the unavailable or failed dependency and state the most useful
525
+ safe recovery step.
526
+ - Actual: summaries show aggregate counts and artifact paths only.
527
+ - Product impact: the developer must open JSON before knowing what to do next.
528
+ - Evidence: captures `003` and `007`.
529
+
530
+ ### Future scope
531
+
532
+ - Cross-model comparison once a second valid model configuration exists.
533
+ - Explicit framework/core/small/internal/transitive classification.
534
+ - Historical trend presentation using the existing stable identities.
535
+
536
+ ## 9. Over-engineering Assessment
537
+
538
+ ### Complexity that should remain
539
+
540
+ - strict schemas and invariant validation;
541
+ - exact-byte lineage and canonical identities;
542
+ - atomic private artifact writers;
543
+ - occurrence-level identity;
544
+ - fail-closed provider and trust behavior;
545
+ - qualification integrity and exact runtime matching;
546
+ - deterministic progress and controlled cancellation;
547
+ - evidence freshness and cache state.
548
+
549
+ These controls directly prevent unsupported recommendations and corrupted
550
+ handoffs.
551
+
552
+ ### Complexity that should be hidden from the default user
553
+
554
+ - qualification IDs and identity digests;
555
+ - internal reason codes when a plain-language next step exists;
556
+ - evaluation, scorecard, benchmark, conformance, and governance commands in
557
+ the primary help path;
558
+ - the eight-artifact implementation chain.
559
+
560
+ The default CLI should present decisions, reasons, risks, and next steps, then
561
+ link to diagnostic details.
562
+
563
+ ### Complexity not yet proving product value
564
+
565
+ Migration Planning has extensive schemas, qualification policy, prompt
566
+ identity, evaluation criteria, trust validation, and artifact lineage. In the
567
+ representative workflow this complexity produced zero grounded actions and a
568
+ handoff that still required full repository investigation.
569
+
570
+ This does not justify removing the safety architecture. It means default
571
+ enablement and further abstraction should wait until the product can produce
572
+ a useful, evidence-bounded roadmap on representative repositories.
573
+
574
+ ## 10. Release Recommendation
575
+
576
+ **`NO_GO`**
577
+
578
+ Automated regression, packaging, integrity, privacy, and provider
579
+ qualification gates pass. Those results establish engineering safety; they do
580
+ not establish product alignment.
581
+
582
+ For a v0.5.0 release positioned as full dependency-upgrade intelligence with
583
+ AI Migration Planning, the real workflow must first:
584
+
585
+ 1. know the actual baseline;
586
+ 2. state whether to upgrade, keep, or investigate;
587
+ 3. avoid non-impact claims outside analyzer coverage;
588
+ 4. produce an implementation-usable bounded roadmap; and
589
+ 5. signal partial provider failure correctly.
590
+
591
+ Migration Checklist should remain experimental.
592
+
593
+ ## 11. Next Tasks
594
+
595
+ 1. Resolve exact current versions from supported lockfiles and environments,
596
+ then prove React and `remark-gfm` already-at-target decisions.
597
+ 2. Add a deterministic decision layer and outcome-first console/Markdown
598
+ summary: upgrade now, plan, keep, insufficient evidence, or not analyzed.
599
+ 3. Make repository-impact conclusions coverage-aware by ecosystem; never emit
600
+ dependency-not-used/not-impacted when the language has no analyzer.
601
+ 4. Define and validate a bounded handoff roadmap containing evidence-supported
602
+ actions, areas, verification, approvals, and ordering; keep Migration
603
+ Checklist experimental until it passes a real-project handoff test.
604
+ 5. Make partial provider/output failures explicit in exit semantics and CLI
605
+ recovery guidance, while retaining successful occurrence results.