@thomasminh1995/depverdict 0.6.0-alpha.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +518 -0
- package/bin/depverdict.js +7 -0
- package/bin/upgradelens.js +7 -0
- package/docs/GR-01-Semantic-Grounding-Failure-Analysis.md +309 -0
- package/docs/GR-02-Versioned-Action-Evaluation-Criteria.md +187 -0
- package/docs/GR-03-Extractive-Contract-Safety-Experiment.md +242 -0
- package/docs/GR-04-Versioned-Production-Extractive-Contract.md +154 -0
- package/docs/IA-01-Repository-Usage-Discovery.md +122 -0
- package/docs/IA-02-Repository-Impact-Analysis.md +118 -0
- package/docs/IA-03-Repository-Impact-Evidence.md +160 -0
- package/docs/IA-04-CLI-Orchestration.md +235 -0
- package/docs/IA-05-Real-Provider-Validation.md +235 -0
- package/docs/IA-05-VinGrade-Validation.md +339 -0
- package/docs/MVP-01.md +48 -0
- package/docs/MVP-02-Architecture.md +536 -0
- package/docs/MVP-02-CLI-HTTP-Runtime.md +50 -0
- package/docs/MVP-02-HTTP-Lifecycle.md +37 -0
- package/docs/MVP-02-Knowledge-Manifest-Generation.md +75 -0
- package/docs/MVP-02-Knowledge-Manifest.md +279 -0
- package/docs/MVP-02-Knowledge-Research-Orchestration.md +62 -0
- package/docs/MVP-02-Knowledge-Store.md +87 -0
- package/docs/MVP-02-PyPI-Registry-Adapter.md +97 -0
- package/docs/MVP-02-Research-Planning.md +196 -0
- package/docs/MVP-02-Source-Provenance.md +81 -0
- package/docs/MVP-02-npm-Registry-Adapter.md +99 -0
- package/docs/RR-01-End-to-End-and-Real-Provider-Validation.md +393 -0
- package/docs/RR-01-RERUN-Extractive-Contract-Validation.md +566 -0
- package/docs/RR-02-Full-Product-Workflow-and-Developer-CLI-UX-Review.md +35 -0
- package/docs/RR02-FIX-01-Persistent-Qualification-Resolution.md +32 -0
- package/docs/RR02-FIX-02-Stage-aware-CLI-Progress-and-Heartbeat.md +28 -0
- package/docs/RR02-FIX-03-npm-Capture-Evidence-Exclusion.md +33 -0
- package/docs/RR02-FIX-03A-Complete-Package-Exclusion-and-Evidence-Commit.md +223 -0
- package/docs/RR02-FIX-04-Event-loop-safe-Heartbeat.md +393 -0
- package/docs/RR02-FIX-05-Materialize-Persisted-Qualification.md +207 -0
- package/docs/RR02-RERUN-CLI-Qualification-Progress-UX-and-Package-Validation.md +342 -0
- package/docs/VinGrade-MVP-02-Validation.md +480 -0
- package/docs/VinGrade-RC02-Live-Validation.md +273 -0
- package/docs/ai-capability-discovery.md +573 -0
- package/docs/ai-engineering-production-readiness.md +390 -0
- package/docs/ai-engineering-review.md +251 -0
- package/docs/ai-runtime-governance-discovery.md +754 -0
- package/docs/architecture-overview.md +78 -0
- package/docs/cli-progress.md +111 -0
- package/docs/decisions/diff-01-brand-distribution-identity.md +450 -0
- package/docs/decisions/diff-02-identity-compatibility-contract.md +274 -0
- package/docs/decisions/diff-03-repository-docs-community-migration.md +165 -0
- package/docs/decisions/diff-04-release-evidence-gap-acceptance.md +110 -0
- package/docs/discovery/mvp-05-ai-migration-planning-discovery.md +350 -0
- package/docs/gateway-runtime-discovery.md +614 -0
- package/docs/live-ai-validation.md +299 -0
- package/docs/migration-planning-qualification-resolution.md +99 -0
- package/docs/migrations/upgradelens-to-depverdict.md +97 -0
- package/docs/mp-r03-deterministic-upgrade-decision-architecture.md +135 -0
- package/docs/mp-r04-evidence-bounded-migration-handoff-architecture.md +116 -0
- package/docs/mp-r05-product-completion-and-decision-first-cli-architecture.md +162 -0
- package/docs/mvp-05-deterministic-context-runtime.md +127 -0
- package/docs/mvp-05-migration-checklist-contract.md +166 -0
- package/docs/mvp-05-migration-checklist-orchestration.md +105 -0
- package/docs/mvp-05-migration-evaluation-and-qualification.md +135 -0
- package/docs/mvp-05-provider-neutral-generator.md +79 -0
- package/docs/ollama-local-smoke-validation.md +156 -0
- package/docs/openai-compatible-runtime-discovery.md +789 -0
- package/docs/openrouter-one-dependency-validation.md +205 -0
- package/docs/oss-02-package-guard-hardening-architecture.md +131 -0
- package/docs/oss-04-public-ci-package-metadata-architecture.md +142 -0
- package/docs/package-content-policy.md +98 -0
- package/docs/releases/v0.5.0-technical-preview.md +114 -0
- package/docs/releases/v0.6.0-alpha.1-depverdict-preview.md +101 -0
- package/docs/reviews/diff-02-identity-contract-compatibility.md +413 -0
- package/docs/reviews/diff-03-repository-docs-community-migration.md +364 -0
- package/docs/reviews/diff-04-depverdict-distribution-identity-readiness-rereview.md +453 -0
- package/docs/reviews/diff-04-fix-post-rename-identity-release-remediation.md +310 -0
- package/docs/reviews/diff-05-final-preview-distribution-qualification.md +561 -0
- package/docs/reviews/mvp-05-final-product-value-workflow-rereview.md +471 -0
- package/docs/reviews/mvp-05-product-workflow-review.md +605 -0
- package/docs/reviews/oss-01-duplicate-artifact-investigation-cleanup.md +411 -0
- package/docs/reviews/oss-02-package-guard-hardening.md +303 -0
- package/docs/reviews/oss-03-community-scaffolding.md +378 -0
- package/docs/reviews/oss-04-public-ci-package-metadata.md +439 -0
- package/docs/reviews/oss-05-technical-preview-qualification.md +482 -0
- package/docs/reviews/upgradelens-vs-upgradedepdetective-source-comparison.md +355 -0
- package/docs/reviews/v0.5.0-pre-release-smoke.md +263 -0
- package/docs/reviews/v0.5.0-version-bump-release-verification.md +306 -0
- package/docs/runtime-contract-discovery.md +521 -0
- package/docs/structured-output-compatibility-report.md +100 -0
- package/docs/ts-fix-01-exact-duplicate-occurrence-target-selection-architecture.md +111 -0
- package/docs/version-analysis-architecture.md +827 -0
- package/eval/README.md +86 -0
- package/eval/datasets/generic/declared-constraint.json +59 -0
- package/eval/datasets/generic/evidence-conflict.json +73 -0
- package/eval/datasets/generic/major-breaking-release.json +66 -0
- package/eval/datasets/generic/missing-evidence.json +46 -0
- package/eval/datasets/generic/patch-release-low.json +59 -0
- package/eval/datasets/node/axios-patch-low.json +59 -0
- package/eval/datasets/node/react-major-breaking.json +66 -0
- package/eval/datasets/node/react-minor-compatibility.json +66 -0
- package/eval/datasets/python/fastapi-deprecation.json +66 -0
- package/eval/datasets/python/pydantic-major-breaking.json +66 -0
- package/eval/migration-planning/golden-dataset-v2.json +306 -0
- package/eval/migration-planning/golden-dataset.json +214 -0
- package/eval/schemas/expected-result.schema.json +88 -0
- package/eval/schemas/golden-case.schema.json +181 -0
- package/package.json +57 -0
- package/schemas/ai-scorecard.schema.json +132 -0
- package/schemas/benchmark-report.schema.json +189 -0
- package/schemas/benchmark.schema.json +55 -0
- package/schemas/capability-profile.schema.json +44 -0
- package/schemas/conformance-report.schema.json +150 -0
- package/schemas/deployment-profile.schema.json +64 -0
- package/schemas/evaluation-report.schema.json +158 -0
- package/schemas/knowledge-evidence-bundle.schema.json +150 -0
- package/schemas/knowledge-manifest.schema.json +548 -0
- package/schemas/metrics.schema.json +178 -0
- package/schemas/migration-checklist-extractive-candidate.schema.json +42 -0
- package/schemas/migration-checklist.schema.json +706 -0
- package/schemas/migration-evaluation-dataset-v2.schema.json +208 -0
- package/schemas/migration-evaluation-dataset.schema.json +204 -0
- package/schemas/migration-planning-qualification-record.schema.json +234 -0
- package/schemas/project-manifest.schema.json +308 -0
- package/schemas/qualification-record.schema.json +56 -0
- package/schemas/repository-impact-evidence.schema.json +232 -0
- package/schemas/repository-impact.schema.json +202 -0
- package/schemas/upgrade-decision.schema.json +273 -0
- package/schemas/usage-index.schema.json +179 -0
- package/schemas/version-analysis.schema.json +449 -0
- package/src/ai-runtime-debug.js +325 -0
- package/src/ai-runtime-error.js +42 -0
- package/src/ai-runtime.js +174 -0
- package/src/ai-scorecard.js +204 -0
- package/src/ai-version-analysis.js +484 -0
- package/src/artifact-root-compatibility.js +91 -0
- package/src/benchmark-report.js +111 -0
- package/src/benchmark-runner.js +191 -0
- package/src/canonical-json.js +69 -0
- package/src/cli.js +1299 -0
- package/src/conformance-report.js +158 -0
- package/src/conformance-runner.js +253 -0
- package/src/constants.js +73 -0
- package/src/cooperative-scheduler.js +79 -0
- package/src/dependencies.js +44 -0
- package/src/dependency-ai-context.js +625 -0
- package/src/detectors.js +253 -0
- package/src/discovery.js +234 -0
- package/src/ecosystem-version-adapter.js +294 -0
- package/src/environment-compatibility.js +77 -0
- package/src/evaluation-comparator.js +158 -0
- package/src/evaluation-report.js +76 -0
- package/src/evaluation-runner.js +248 -0
- package/src/evidence-source-adapter.js +472 -0
- package/src/files.js +89 -0
- package/src/governance-diagnostics.js +64 -0
- package/src/governance-loader.js +63 -0
- package/src/governance-metadata.js +346 -0
- package/src/governance-validator.js +360 -0
- package/src/http/bounded-fetch.js +278 -0
- package/src/http/cli-http-runtime.js +44 -0
- package/src/impact/input-loader.js +157 -0
- package/src/impact/matcher.js +40 -0
- package/src/impact/repository-impact.js +199 -0
- package/src/impact/runtime.js +24 -0
- package/src/impact/status.js +62 -0
- package/src/impact/writer.js +30 -0
- package/src/impact-evidence/input-loader.js +202 -0
- package/src/impact-evidence/repository-impact-evidence.js +234 -0
- package/src/impact-evidence/runtime.js +16 -0
- package/src/impact-evidence/writer.js +30 -0
- package/src/index.js +563 -0
- package/src/installed-version-baseline.js +196 -0
- package/src/knowledge-cache.js +324 -0
- package/src/knowledge-evidence-bundle.js +101 -0
- package/src/knowledge-evidence-producer.js +233 -0
- package/src/knowledge-manifest-builder.js +188 -0
- package/src/knowledge-manifest-writer.js +32 -0
- package/src/knowledge-manifest.js +255 -0
- package/src/knowledge-research.js +615 -0
- package/src/metrics-engine.js +205 -0
- package/src/migration-checklist/ai-candidate.js +320 -0
- package/src/migration-checklist/assembler.js +37 -0
- package/src/migration-checklist/context-runtime.js +828 -0
- package/src/migration-checklist/evaluation/action-criteria.js +244 -0
- package/src/migration-checklist/evaluation/comparator-v2.js +332 -0
- package/src/migration-checklist/evaluation/comparator.js +279 -0
- package/src/migration-checklist/evaluation/dataset-v2.js +227 -0
- package/src/migration-checklist/evaluation/dataset.js +336 -0
- package/src/migration-checklist/evaluation/extractive-fixtures-v2.js +148 -0
- package/src/migration-checklist/evaluation/metrics-v2.js +226 -0
- package/src/migration-checklist/evaluation/metrics.js +158 -0
- package/src/migration-checklist/evaluation/qualification-v2.js +321 -0
- package/src/migration-checklist/evaluation/qualification.js +239 -0
- package/src/migration-checklist/evaluation/runner-v2.js +294 -0
- package/src/migration-checklist/evaluation/runner.js +194 -0
- package/src/migration-checklist/evaluation/scorecard-v2.js +106 -0
- package/src/migration-checklist/evaluation/scorecard.js +86 -0
- package/src/migration-checklist/extractive-candidate.js +166 -0
- package/src/migration-checklist/extractive-prompt.js +62 -0
- package/src/migration-checklist/generator.js +702 -0
- package/src/migration-checklist/grounding-policy.js +117 -0
- package/src/migration-checklist/input-loader.js +613 -0
- package/src/migration-checklist/migration-checklist.js +635 -0
- package/src/migration-checklist/presentation.js +292 -0
- package/src/migration-checklist/progress.js +96 -0
- package/src/migration-checklist/prompt.js +83 -0
- package/src/migration-checklist/qualification-guard.js +462 -0
- package/src/migration-checklist/qualification-resolution.js +122 -0
- package/src/migration-checklist/qualification-store.js +225 -0
- package/src/migration-checklist/runtime.js +205 -0
- package/src/migration-checklist/verification.js +134 -0
- package/src/migration-checklist/writer.js +35 -0
- package/src/openai-compatible-provider.js +451 -0
- package/src/orchestration/failure-log.js +32 -0
- package/src/orchestration/pipeline.js +200 -0
- package/src/orchestration/progress-events.js +337 -0
- package/src/orchestration/progress-reporter.js +131 -0
- package/src/orchestration/text-writer.js +22 -0
- package/src/portable.js +13 -0
- package/src/product-completion.js +249 -0
- package/src/project-manifest-input.js +90 -0
- package/src/project-manifest.js +141 -0
- package/src/python-requirements.js +137 -0
- package/src/registry/npm-packument.js +256 -0
- package/src/registry/npm-registry-adapter.js +262 -0
- package/src/registry/pypi-project.js +300 -0
- package/src/registry/pypi-registry-adapter.js +235 -0
- package/src/registry/sanitize-registry-body.js +51 -0
- package/src/renderers/console.js +160 -0
- package/src/renderers/impact-presentation.js +278 -0
- package/src/renderers/markdown.js +172 -0
- package/src/research-plan.js +455 -0
- package/src/runtime-conformance.js +275 -0
- package/src/source-provenance.js +393 -0
- package/src/source-url.js +62 -0
- package/src/structured-output-schema.js +66 -0
- package/src/target-selector.js +306 -0
- package/src/upgrade-decision/input-loader.js +107 -0
- package/src/upgrade-decision/presentation.js +43 -0
- package/src/upgrade-decision/runtime.js +21 -0
- package/src/upgrade-decision/upgrade-decision.js +626 -0
- package/src/upgrade-decision/writer.js +30 -0
- package/src/usage/analyzer-registry.js +63 -0
- package/src/usage/coverage.js +116 -0
- package/src/usage/input-loader.js +139 -0
- package/src/usage/js/analyzer.js +240 -0
- package/src/usage/js/parser.js +21 -0
- package/src/usage/runtime.js +187 -0
- package/src/usage/scope.js +44 -0
- package/src/usage/source-files.js +50 -0
- package/src/usage/usage-index.js +217 -0
- package/src/usage/writer.js +31 -0
- package/src/version-analysis-loader.js +203 -0
- package/src/version-analysis-manifest.js +314 -0
- package/src/version-analysis-writer.js +30 -0
|
@@ -0,0 +1,605 @@
|
|
|
1
|
+
# UpgradeLens MVP-05 — Full Product Workflow and Feature Alignment Review
|
|
2
|
+
|
|
3
|
+
## 1. Executive Verdict
|
|
4
|
+
|
|
5
|
+
**`PARTIAL_MATCH`**
|
|
6
|
+
|
|
7
|
+
The UpgradeLens pipeline is operational, deterministic, well validated, and
|
|
8
|
+
safe against unsupported AI actions. It can discover a representative
|
|
9
|
+
polyglot repository, research public evidence, run provider-backed Version
|
|
10
|
+
Analysis, index JavaScript/TypeScript usage, preserve artifact lineage, and
|
|
11
|
+
produce a human-review-gated report.
|
|
12
|
+
|
|
13
|
+
It does not yet deliver the complete product outcome claimed for MVP-05. The
|
|
14
|
+
real VinGrade workflow produced no explicit upgrade recommendation, no
|
|
15
|
+
grounded migration action, no candidate location, no ordered roadmap, and no
|
|
16
|
+
verification or recovery plan. A Coding Agent had to inspect lockfiles, source,
|
|
17
|
+
tests, and runtime configuration to derive even “already at target” and “hold
|
|
18
|
+
the upgrade” conclusions.
|
|
19
|
+
|
|
20
|
+
| Decision | Result |
|
|
21
|
+
| --- | --- |
|
|
22
|
+
| v0.5.0 release readiness | **`NO_GO`** for the stated full dependency-upgrade intelligence and migration-roadmap positioning |
|
|
23
|
+
| Migration Checklist readiness | **`KEEP_EXPERIMENTAL`**; not ready for default enablement |
|
|
24
|
+
| Real-provider qualification | **`QUALIFIED`** for `openai-compatible / openai/gpt-5.5 / openai-compatible` |
|
|
25
|
+
| Product workflow readiness | Pipeline-ready; product outcome **partially ready** |
|
|
26
|
+
| Coding Agent handoff readiness | **Not ready for implementation**; useful only to start investigation |
|
|
27
|
+
|
|
28
|
+
This recommendation is based on product-output gaps reproduced on a real
|
|
29
|
+
repository, not on automated test failures.
|
|
30
|
+
|
|
31
|
+
## 2. Environment and Scope
|
|
32
|
+
|
|
33
|
+
| Item | Value |
|
|
34
|
+
| --- | --- |
|
|
35
|
+
| UpgradeLens branch | `feat/mvp-05-evidence-migration-checklist` |
|
|
36
|
+
| UpgradeLens commit | `44eb942e21ea78550700990a00bd74635ce48bcc` (`fix: event-loop starvation`) |
|
|
37
|
+
| Package | `upgradelens@0.4.0` |
|
|
38
|
+
| Node.js | `v26.0.0` |
|
|
39
|
+
| npm | `11.12.1` |
|
|
40
|
+
| Platform | macOS arm64 |
|
|
41
|
+
| Representative repository | VinGrade |
|
|
42
|
+
| VinGrade commit | `25811ef997fcf45810105e89e4500688f28f7ba5` |
|
|
43
|
+
| VinGrade execution target | Clean isolated `git archive` snapshot |
|
|
44
|
+
| Provider/model | `openai-compatible` / `openai/gpt-5.5` |
|
|
45
|
+
| Provider debug output | Disabled |
|
|
46
|
+
| Qualification | Persisted, exact identity, `QUALIFIED` |
|
|
47
|
+
|
|
48
|
+
The original VinGrade checkout had a deleted tracked `Makefile` and unrelated
|
|
49
|
+
untracked user files. It was not modified. All workflow writes went to
|
|
50
|
+
isolated temporary snapshots.
|
|
51
|
+
|
|
52
|
+
The review covered the current source, schemas, tests, fixtures, public CLI,
|
|
53
|
+
packaged CLI, full online and offline workflows, failure handling, occurrence
|
|
54
|
+
identity, provider qualification, console/Markdown presentation, and Coding
|
|
55
|
+
Agent handoff.
|
|
56
|
+
|
|
57
|
+
### Validation status
|
|
58
|
+
|
|
59
|
+
| Validation | Result |
|
|
60
|
+
| --- | --- |
|
|
61
|
+
| Canonical `npm run check` | 545 passed, 0 failed, 1 skipped |
|
|
62
|
+
| Skip | Local-loopback lifecycle test: sandbox does not expose listeners to that test |
|
|
63
|
+
| Package guard | 209 files after this report, 0 CLI-capture evidence, 15/15 required assets |
|
|
64
|
+
| Static lint | Blocked: repository defines no lint script |
|
|
65
|
+
| Schema validation | Passed through automated suite and runtime artifact writers/loaders |
|
|
66
|
+
| `git diff --check` | Passed |
|
|
67
|
+
| Fresh npm pack | Passed |
|
|
68
|
+
| Clean offline install | Passed using exact local dependency tarballs |
|
|
69
|
+
| Packaged version/help/import | Passed; version `0.4.0`, 408 public exports |
|
|
70
|
+
| Second-model comparison | Blocked: no second model configuration was available |
|
|
71
|
+
| Provider token/cost comparison | Blocked: production artifacts do not retain request usage/cost totals |
|
|
72
|
+
|
|
73
|
+
The canonical suite’s single skip is reported rather than counted as a pass.
|
|
74
|
+
No test, schema, fixture, timeout, policy, or source implementation was changed.
|
|
75
|
+
|
|
76
|
+
## 3. Full Workflow Result
|
|
77
|
+
|
|
78
|
+
The main command was:
|
|
79
|
+
|
|
80
|
+
```text
|
|
81
|
+
upgradelens analyze . --progress plain --experimental-migration-checklist
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
The command ran from the isolated VinGrade snapshot. Public registry research
|
|
85
|
+
used fresh or cached evidence. Version Analysis and the eligible Migration
|
|
86
|
+
Checklist context used the configured real provider.
|
|
87
|
+
|
|
88
|
+
| Stage | Status | Input | Output | Duration | User-visible result |
|
|
89
|
+
| --- | --- | --- | --- | ---: | --- |
|
|
90
|
+
| Project Discovery | Passed | VinGrade source tree | `project-manifest.json` | 0.1s | 2 projects; Node and Python |
|
|
91
|
+
| Knowledge Research | Passed with warnings | 47 occurrences / 46 packages | `knowledge-manifest.json`, `knowledge-evidence-bundle.json` | 9.3s | Not shown in final summary |
|
|
92
|
+
| Version Analysis | Passed with partial results | Project, knowledge, 577 evidence records | `version-analysis.json` | 398.8s | Final summary showed 36 analyzed, 3 skipped, 8 failed |
|
|
93
|
+
| Repository Usage Discovery | Passed with warning | Source tree | `usage-index.json` | 0.2s | 17 dependencies, 152 symbols, 79 files; detail not shown in final summary |
|
|
94
|
+
| Repository Impact Analysis | Passed | Version Analysis and Usage Index | `repository-impact.json` | <0.1s | 0 impacted, 36 not impacted, 11 not analyzed |
|
|
95
|
+
| Repository Impact Evidence | Passed | Impact and lineage artifacts | `repository-impact-evidence.json` | <0.1s | 4 finding records, no affected files |
|
|
96
|
+
| Migration Checklist | Passed but incomplete | Full lineage and exact qualification | `migration-checklist.json` | 0.2s | 4 manual-review items, 0 grounded actions, 0 locations |
|
|
97
|
+
| Markdown Report | Passed | Presentation view models | `repository-impact.md` | <0.1s | Report path shown |
|
|
98
|
+
|
|
99
|
+
Total CLI time was 408.6 seconds; the capture wrapper measured 409.3 seconds.
|
|
100
|
+
All eight stages reached terminal success, but business completeness was
|
|
101
|
+
`INCOMPLETE`.
|
|
102
|
+
|
|
103
|
+
### Artifact results
|
|
104
|
+
|
|
105
|
+
- Project Discovery: 2 projects, 47 dependency occurrences.
|
|
106
|
+
- Knowledge Research: 45 resolved packages, 1 unavailable package, 298
|
|
107
|
+
sources, 577 evidence records, 84 manifest warnings.
|
|
108
|
+
- Version Analysis: 36 analyzed, 3 skipped, 8 failed, 46 requiring review.
|
|
109
|
+
- Risk: 1 medium, 46 unknown.
|
|
110
|
+
- Usage: JavaScript/TypeScript coverage only; 1 parse warning.
|
|
111
|
+
- Impact: 4 breaking findings, none reported as impacted.
|
|
112
|
+
- Checklist: 47 dependency records, 4 finding records, 4 deterministic
|
|
113
|
+
manual-review items, no grounded action and no candidate location.
|
|
114
|
+
|
|
115
|
+
### Provider requests
|
|
116
|
+
|
|
117
|
+
The one-call-per-eligible-context contract and retained result states show:
|
|
118
|
+
|
|
119
|
+
- 44 Version Analysis attempts: 36 successful and 8 HTTP 402
|
|
120
|
+
`INSUFFICIENT_CREDIT` failures;
|
|
121
|
+
- 3 skipped Version Analysis contexts made no request;
|
|
122
|
+
- 1 eligible Migration Checklist attempt failed and was contained as
|
|
123
|
+
`AI_RUNTIME_FAILED`;
|
|
124
|
+
- total real-provider attempts: **45**;
|
|
125
|
+
- hidden retries: **0**.
|
|
126
|
+
|
|
127
|
+
A separate loopback invalid-output fixture received exactly one simulated
|
|
128
|
+
request. It is not counted as a real-provider request.
|
|
129
|
+
|
|
130
|
+
## 4. Core Feature Alignment Matrix
|
|
131
|
+
|
|
132
|
+
| Feature | Verdict | Evidence | Developer/team impact | v0.5.0 relevance |
|
|
133
|
+
| --- | --- | --- | --- | --- |
|
|
134
|
+
| Dependency Inventory | **MATCH** | Project Manifest preserved 47 occurrences across Node and Python with project, manifest, declaration, and dependency type | Team can enumerate direct declarations and distinguish occurrences | Required and ready |
|
|
135
|
+
| Upgrade Detection | **PARTIAL_MATCH** | Registry targets and missing-target states exist, but every resolvable declaration is analyzed; there is no separate “requires attention” selection | Teams receive broad analysis rather than a focused attention list | Required improvement |
|
|
136
|
+
| Upgrade Recommendation | **NOT_MATCH** | Version Analysis has status, risk, review reasons, and `nextAction`, but no `upgrade now / plan / keep / insufficient evidence` decision | A tech lead must infer whether any dependency should change | Release-critical |
|
|
137
|
+
| Reason and Evidence | **PARTIAL_MATCH** | Findings reference exact evidence IDs and target versions; output also clearly labels registry latest as a fact, not a recommendation | Good auditability, but reasons do not resolve the decision | Important |
|
|
138
|
+
| Repository Risk Assessment | **PARTIAL_MATCH** | JS/TS exact-symbol matching works; Python is not indexed, yet one LangSmith finding was reported `DEPENDENCY_NOT_USED` and `NOT_IMPACTED` despite real Python usage | False confidence is possible outside supported analyzer coverage | Release-critical |
|
|
139
|
+
| Dependency Classification | **NOT_MATCH** | Dependency type is recorded, but no framework/core, small package, internal package, or transitive classification exists | React and `remark-gfm` receive essentially the same decision treatment | Important |
|
|
140
|
+
| Prioritization | **NOT_MATCH** | No dependency or action priority, urgency, rank, or planning order is presented | Teams cannot turn the report into a work queue | Release-critical |
|
|
141
|
+
| Migration Roadmap | **NOT_MATCH** | Checklist had zero grounded actions/locations; the production prompt forbids prerequisites, order, rollback, and verification claims | The output is a review ledger, not a migration roadmap | Release-critical |
|
|
142
|
+
| Human Review | **MATCH** | 46/47 Version results required review; checklist always states human review is mandatory | Unsafe automation is prevented | Required and ready |
|
|
143
|
+
| Coding Agent Handoff | **PARTIAL_MATCH** | Stable IDs, evidence refs, findings, and limitations help investigation; the Agent still had to recover versions, files, tests, and runtime constraints from source | Agent cannot safely begin implementation from the artifact | Release-critical |
|
|
144
|
+
| Model Transparency | **PARTIAL_MATCH** | Checklist console shows qualification and runtime tuple; Version Analysis and Checklist JSON do not persist provider/model/request identity | Saved artifacts cannot independently establish which runtime produced results | Important |
|
|
145
|
+
| Evidence Freshness | **MATCH** | Source records retain `retrievedAt`, freshness, publication dates, and cache results | Reviewer can distinguish fresh, cached, and unavailable evidence | Required and ready |
|
|
146
|
+
| Offline Readiness | **MATCH** | Empty-cache offline run safely produced 47 not-analyzed results; fresh-cache offline research reused 45 packages without network | Offline limits are representable without inventing latest facts | Ready with UX caveat |
|
|
147
|
+
| Continuous Tracking Readiness | **MATCH** | Canonical IDs, schema versions, artifact digests, research IDs, context IDs, and lineage are stable | Future runs can be compared without name-only matching | Ready |
|
|
148
|
+
| CLI-first Delivery | **PARTIAL_MATCH** | One `analyze` command runs the pipeline with stable progress and CI-friendly plain mode; important decisions and failure next steps remain buried in artifacts | Operationally usable, but not yet outcome-first | Important |
|
|
149
|
+
|
|
150
|
+
**Count: 5 MATCH, 6 PARTIAL_MATCH, 4 NOT_MATCH.**
|
|
151
|
+
|
|
152
|
+
## 5. Three Core Questions Assessment
|
|
153
|
+
|
|
154
|
+
### Question 1 — Should the dependency be upgraded?
|
|
155
|
+
|
|
156
|
+
**No clear answer is produced.**
|
|
157
|
+
|
|
158
|
+
React is the decisive example:
|
|
159
|
+
|
|
160
|
+
- declared constraint: `^19.2.6`;
|
|
161
|
+
- lockfile installed version: `19.2.7`;
|
|
162
|
+
- reported target: `19.2.7`;
|
|
163
|
+
- Version Analysis current version: `null`;
|
|
164
|
+
- risk: `unknown`;
|
|
165
|
+
- next action: review before impact analysis.
|
|
166
|
+
|
|
167
|
+
The technically useful answer is “already at the target; do not perform a
|
|
168
|
+
version change.” UpgradeLens does not read the lockfile baseline and therefore
|
|
169
|
+
cannot produce that decision.
|
|
170
|
+
|
|
171
|
+
The small package `remark-gfm` is also locked at the reported `4.0.1` target,
|
|
172
|
+
but receives unknown risk and mandatory review instead of a concise “keep
|
|
173
|
+
current version” outcome.
|
|
174
|
+
|
|
175
|
+
### Question 2 — Why, and what is the risk?
|
|
176
|
+
|
|
177
|
+
**Partially answered.**
|
|
178
|
+
|
|
179
|
+
Strengths:
|
|
180
|
+
|
|
181
|
+
- evidence references are exact and target-scoped;
|
|
182
|
+
- uncertainty and registry-latest limitations are explicit;
|
|
183
|
+
- findings are retained separately from deterministic impact matching;
|
|
184
|
+
- provider failure does not invent findings.
|
|
185
|
+
|
|
186
|
+
Weaknesses:
|
|
187
|
+
|
|
188
|
+
- the unresolved baseline prevents meaningful delta risk for most packages;
|
|
189
|
+
- 46/47 results remain `unknown`;
|
|
190
|
+
- no framework/core versus small-package context changes the presentation;
|
|
191
|
+
- Python usage is unsupported, but the report still says LangSmith is not
|
|
192
|
+
impacted because the dependency is not used.
|
|
193
|
+
|
|
194
|
+
A tech lead must inspect multiple JSON artifacts and repository source to
|
|
195
|
+
understand the actual risk.
|
|
196
|
+
|
|
197
|
+
### Question 3 — What is the migration roadmap?
|
|
198
|
+
|
|
199
|
+
**Not answered.**
|
|
200
|
+
|
|
201
|
+
The checklist contains:
|
|
202
|
+
|
|
203
|
+
- 4 deterministic manual-review placeholders;
|
|
204
|
+
- 0 grounded actions;
|
|
205
|
+
- 0 AI-selected official guidance items;
|
|
206
|
+
- 0 candidate locations;
|
|
207
|
+
- no order, preconditions, verification strategy, rollout, or recovery step.
|
|
208
|
+
|
|
209
|
+
The extractive production contract intentionally prevents invented actions,
|
|
210
|
+
which is the correct safety behavior. It also means the current artifact is
|
|
211
|
+
not the migration roadmap described by the product positioning.
|
|
212
|
+
|
|
213
|
+
### Planning-meeting usability
|
|
214
|
+
|
|
215
|
+
The report is useful as a safe evidence and uncertainty inventory. It is not
|
|
216
|
+
ready to drive an upgrade-planning meeting without additional lockfile,
|
|
217
|
+
source, and test investigation.
|
|
218
|
+
|
|
219
|
+
## 6. Scenario Results
|
|
220
|
+
|
|
221
|
+
### Scenario A — Representative real project
|
|
222
|
+
|
|
223
|
+
**Result: PARTIAL_MATCH**
|
|
224
|
+
|
|
225
|
+
VinGrade is a real polyglot project with 47 occurrences, 46 packages, a React
|
|
226
|
+
frontend, Python backend, core and small packages, and real source usage.
|
|
227
|
+
|
|
228
|
+
The main command correctly ran all stages in order and produced the complete
|
|
229
|
+
artifact chain. Critical loader/lineage failures stop their commands.
|
|
230
|
+
Package-local provider failures are retained without deleting successful
|
|
231
|
+
results.
|
|
232
|
+
|
|
233
|
+
The workflow still ends with no recommendation and no roadmap.
|
|
234
|
+
|
|
235
|
+
### Scenario B — Core framework: React
|
|
236
|
+
|
|
237
|
+
**Result: NOT_MATCH**
|
|
238
|
+
|
|
239
|
+
React is already locked at the reported target `19.2.7`. UpgradeLens treats
|
|
240
|
+
the baseline as unresolved, assigns unknown risk, and produces a private
|
|
241
|
+
React Server Components path finding that does not match repository usage.
|
|
242
|
+
It does not say “already current,” propose no change, identify architecture
|
|
243
|
+
areas, or provide framework-level verification.
|
|
244
|
+
|
|
245
|
+
### Scenario C — Small package: `remark-gfm`
|
|
246
|
+
|
|
247
|
+
**Result: NOT_MATCH**
|
|
248
|
+
|
|
249
|
+
`remark-gfm` is also already locked at its target `4.0.1`. The output lists
|
|
250
|
+
type/documentation changes, leaves risk unknown, and requires human review.
|
|
251
|
+
There is no concise keep-current conclusion. Dependency type is preserved,
|
|
252
|
+
but there is no small-package classification or proportional UX.
|
|
253
|
+
|
|
254
|
+
### Scenario D — Insufficient evidence
|
|
255
|
+
|
|
256
|
+
**Result: MATCH for safety; PARTIAL_MATCH for UX**
|
|
257
|
+
|
|
258
|
+
With an empty offline cache:
|
|
259
|
+
|
|
260
|
+
- 46 packages were unavailable;
|
|
261
|
+
- evidence count was 0;
|
|
262
|
+
- all 47 Version Analysis results were skipped;
|
|
263
|
+
- no model was called;
|
|
264
|
+
- all 47 required review;
|
|
265
|
+
- Migration Checklist generated no grounded action.
|
|
266
|
+
|
|
267
|
+
No latest version, command, source location, or migration instruction was
|
|
268
|
+
invented. The final console output does not tell the user to populate the
|
|
269
|
+
cache or rerun research online; that next step remains inside artifacts.
|
|
270
|
+
|
|
271
|
+
### Scenario E — Mixed occurrences
|
|
272
|
+
|
|
273
|
+
**Result: MATCH for identity; PARTIAL_MATCH for presentation**
|
|
274
|
+
|
|
275
|
+
The duplicate `langchain-openai` declarations remained separate:
|
|
276
|
+
|
|
277
|
+
- unversioned occurrence: distinct context/result ID, skipped with
|
|
278
|
+
`BASELINE_UNSUPPORTED`;
|
|
279
|
+
- `>=0.3.0` occurrence: distinct context/result ID, analyzed.
|
|
280
|
+
|
|
281
|
+
No occurrence was silently chosen or merged, and one skipped occurrence did
|
|
282
|
+
not change the other result. Markdown shows two identical
|
|
283
|
+
`langchain-openai` headings without the declaration/context identity, which is
|
|
284
|
+
ambiguous for a human reviewer.
|
|
285
|
+
|
|
286
|
+
### Scenario F — Failure and recovery
|
|
287
|
+
|
|
288
|
+
| Failure | Exit | Result |
|
|
289
|
+
| --- | ---: | --- |
|
|
290
|
+
| Missing upstream Project Manifest | 1 | Correct stop; raw `ENOENT` path, no stage label or concrete next command |
|
|
291
|
+
| Missing AI runtime configuration | 1 | Clear configuration message; does not explain provider/model setup beyond endpoint |
|
|
292
|
+
| Invalid artifact lineage | 1 | Precise digest and repository-name mismatch; no artifact replacement |
|
|
293
|
+
| Invalid structured response | 0 | Fail-closed result artifact with `OUTPUT_JSON_INVALID`, but CLI prints “complete” and exits success |
|
|
294
|
+
| Real provider insufficient credit | 0 for full workflow | 8 package failures retained; final summary shows failure count, but credit recovery is not shown |
|
|
295
|
+
| Migration Checklist provider failure | 0 for full workflow | Contained as manual review; no unsupported action |
|
|
296
|
+
| Pipeline cancellation | 130 | Existing same-HEAD packaged capture proves controlled cancellation and no partial Usage Index |
|
|
297
|
+
|
|
298
|
+
The runtime performs no automatic retry. The invalid-output fixture received
|
|
299
|
+
one request.
|
|
300
|
+
|
|
301
|
+
### Coding Agent handoff
|
|
302
|
+
|
|
303
|
+
**Result: NOT READY**
|
|
304
|
+
|
|
305
|
+
Using only the source repository and Migration Checklist, the Agent concluded:
|
|
306
|
+
|
|
307
|
+
- React, React DOM, and PDF.js were already at target;
|
|
308
|
+
- LangSmith should be held pending API evidence;
|
|
309
|
+
- Node runtime compatibility and real test prerequisites need review.
|
|
310
|
+
|
|
311
|
+
Those conclusions required repository re-research. The checklist itself
|
|
312
|
+
provided no concrete change, test, file, location, or target approval.
|
|
313
|
+
|
|
314
|
+
### Cross-model readiness
|
|
315
|
+
|
|
316
|
+
**Execution blocked.** Only `openai-compatible / openai/gpt-5.5` was
|
|
317
|
+
configured. No Claude Sonnet configuration was available, so no cross-model
|
|
318
|
+
quality claim is made.
|
|
319
|
+
|
|
320
|
+
The qualification resolver distinguishes exact qualified, missing, mismatch,
|
|
321
|
+
corrupted, insufficient, and rejected records. It does not expose a separate
|
|
322
|
+
user-facing compatible/qualified/experimental/rejected/unknown model taxonomy
|
|
323
|
+
in saved Version Analysis artifacts.
|
|
324
|
+
|
|
325
|
+
### Offline readiness
|
|
326
|
+
|
|
327
|
+
**Result: MATCH**
|
|
328
|
+
|
|
329
|
+
The empty-cache offline run safely represented external verification as
|
|
330
|
+
unavailable. A separate offline fresh-cache run reused 45 packages, retained 1
|
|
331
|
+
cache miss, performed no registry request, and preserved evidence freshness.
|
|
332
|
+
|
|
333
|
+
Project-only discovery and usage analysis continue to work offline. The final
|
|
334
|
+
console summary should state the offline evidence limitation more directly.
|
|
335
|
+
|
|
336
|
+
## 7. CLI UX Review
|
|
337
|
+
|
|
338
|
+
### What works
|
|
339
|
+
|
|
340
|
+
- `upgradelens analyze` is the correct single workflow command.
|
|
341
|
+
- Stage order and plain progress are stable and CI-friendly.
|
|
342
|
+
- Heartbeats make the long provider stage visibly active.
|
|
343
|
+
- Final summary clearly shows analyzed/skipped/failed/review counts.
|
|
344
|
+
- Qualification status, source, ID, runtime tuple, and experimental override
|
|
345
|
+
are visible when Migration Checklist runs.
|
|
346
|
+
- Secret-like provider values are not printed.
|
|
347
|
+
- Missing configuration and lineage failures are concise.
|
|
348
|
+
- Output paths are portable relative paths in the success summary.
|
|
349
|
+
|
|
350
|
+
### What is difficult
|
|
351
|
+
|
|
352
|
+
- The summary leads with counts, not “upgrade / keep / investigate” decisions.
|
|
353
|
+
- `--help` describes commands and artifacts, not the three product outcomes.
|
|
354
|
+
- Provider setup is not explained beyond environment variable errors.
|
|
355
|
+
- Research warnings and unavailable-package identity are absent from final
|
|
356
|
+
output.
|
|
357
|
+
- `NOT_IMPACTED` appears without analyzer-coverage context.
|
|
358
|
+
- A package-local provider/output failure can exit 0 and print “complete.”
|
|
359
|
+
- Migration Checklist exposes internal qualification details before any useful
|
|
360
|
+
action.
|
|
361
|
+
- The developer must open multiple JSON artifacts to find next steps and
|
|
362
|
+
limitations.
|
|
363
|
+
- The full help page exposes evaluation, scorecard, benchmark, conformance,
|
|
364
|
+
and governance commands at the same level as the primary workflow.
|
|
365
|
+
|
|
366
|
+
### Recommended default output
|
|
367
|
+
|
|
368
|
+
Lead with a dependency decision table:
|
|
369
|
+
|
|
370
|
+
```text
|
|
371
|
+
Dependency Decision Current → Target Risk Why / next step
|
|
372
|
+
react Keep current 19.2.7 → 19.2.7 Low Already at target
|
|
373
|
+
axios Insufficient evidence 1.17.0 → 1.18.1 Unknown Review release delta
|
|
374
|
+
vite Not analyzed unknown Unknown Collect target evidence
|
|
375
|
+
```
|
|
376
|
+
|
|
377
|
+
Then show affected areas, human approvals, and the report/checklist paths.
|
|
378
|
+
Keep qualification digests and detailed reason codes behind a verbose or
|
|
379
|
+
diagnostic view.
|
|
380
|
+
|
|
381
|
+
### Capture references
|
|
382
|
+
|
|
383
|
+
Captures are stored locally under:
|
|
384
|
+
|
|
385
|
+
```text
|
|
386
|
+
.upgradelens/review/mvp-05-workflow/
|
|
387
|
+
```
|
|
388
|
+
|
|
389
|
+
Important entries:
|
|
390
|
+
|
|
391
|
+
- `003-vingrade-full-offline-qualified/`
|
|
392
|
+
- `007-vingrade-full-online-qualified/`
|
|
393
|
+
- `008-missing-ai-runtime-config/`
|
|
394
|
+
- `009-invalid-artifact-lineage/`
|
|
395
|
+
- `010-invalid-structured-response/`
|
|
396
|
+
- `013-packaged-version/`
|
|
397
|
+
- `014-packaged-help/`
|
|
398
|
+
- `016-agent-handoff-assessment.md`
|
|
399
|
+
- `017-offline-fresh-cache-research/`
|
|
400
|
+
- `018-post-report-npm-pack/`
|
|
401
|
+
- `019-final-package-guard/`
|
|
402
|
+
|
|
403
|
+
Each command entry retains command, working directory, safe runtime identity,
|
|
404
|
+
exit code, exact stdout/stderr, duration, and artifact inventory. Eight
|
|
405
|
+
important final screens were rendered from exact raw transcripts; they are
|
|
406
|
+
not presented as native terminal screenshots.
|
|
407
|
+
|
|
408
|
+
## 8. Defects and Gaps
|
|
409
|
+
|
|
410
|
+
### Blocker
|
|
411
|
+
|
|
412
|
+
#### B-01 — The workflow does not produce an upgrade decision or migration roadmap
|
|
413
|
+
|
|
414
|
+
- Reproduction: run the qualified online VinGrade workflow.
|
|
415
|
+
- Expected: explicit upgrade/plan/keep/insufficient decision plus a safe,
|
|
416
|
+
ordered roadmap for actionable dependencies.
|
|
417
|
+
- Actual: 47 dependencies, 4 breaking findings, 4 manual-review placeholders,
|
|
418
|
+
0 grounded actions, 0 locations, and no roadmap.
|
|
419
|
+
- Product impact: Questions 1 and 3 are unanswered; the headline MVP-05
|
|
420
|
+
product value is not delivered.
|
|
421
|
+
- Evidence: capture `007`; `migration-checklist.json`; Coding Agent handoff.
|
|
422
|
+
|
|
423
|
+
### High
|
|
424
|
+
|
|
425
|
+
#### H-01 — Lockfile baselines are ignored
|
|
426
|
+
|
|
427
|
+
- Reproduction: inspect React, React DOM, PDF.js, and `remark-gfm` in VinGrade.
|
|
428
|
+
- Expected: use the exact package-lock version as current baseline.
|
|
429
|
+
- Actual: current version is `null`; target versions already installed are
|
|
430
|
+
treated as uncertain registry facts.
|
|
431
|
+
- Product impact: unnecessary provider work, unknown risk, and no keep-current
|
|
432
|
+
decision.
|
|
433
|
+
- Evidence: `FE/package-lock.json` versus `version-analysis.json`.
|
|
434
|
+
|
|
435
|
+
#### H-02 — Unsupported Python usage can be presented as not impacted
|
|
436
|
+
|
|
437
|
+
- Reproduction: the LangSmith breaking finding in the online run.
|
|
438
|
+
- Expected: usage coverage unknown because no Python analyzer exists.
|
|
439
|
+
- Actual: `DEPENDENCY_NOT_USED`, `NOT_IMPACTED`; VinGrade imports and uses
|
|
440
|
+
`langsmith.Client`.
|
|
441
|
+
- Product impact: false repository-risk confidence.
|
|
442
|
+
- Evidence: `repository-impact-evidence.json`, VinGrade
|
|
443
|
+
`evaluation/langsmith/*.py`, capture `007`.
|
|
444
|
+
|
|
445
|
+
#### H-03 — Package-level provider/output failures can look successful
|
|
446
|
+
|
|
447
|
+
- Reproduction: invalid structured-response capture and the real provider
|
|
448
|
+
credit exhaustion.
|
|
449
|
+
- Expected: nonzero or explicit partial-success exit with failed package and
|
|
450
|
+
recovery action in CLI output.
|
|
451
|
+
- Actual: exit 0 and “AI Version Analysis complete”; failed result exists only
|
|
452
|
+
in JSON. The full run also exits 0 with 8 failed dependencies.
|
|
453
|
+
- Product impact: CI or a developer can treat incomplete analysis as success.
|
|
454
|
+
- Evidence: captures `007` and `010`.
|
|
455
|
+
|
|
456
|
+
### Medium
|
|
457
|
+
|
|
458
|
+
#### M-01 — Runtime identity is not self-contained in analysis artifacts
|
|
459
|
+
|
|
460
|
+
- Reproduction: inspect the saved Version Analysis and Migration Checklist
|
|
461
|
+
artifacts from capture `007`.
|
|
462
|
+
- Expected: saved provider, model, runtime mode, and qualification status.
|
|
463
|
+
- Actual: console checklist presentation shows that identity, while the JSON
|
|
464
|
+
retains only UpgradeLens generator identity and lineage.
|
|
465
|
+
- Product impact: a later reviewer cannot independently establish which AI
|
|
466
|
+
runtime produced the saved result.
|
|
467
|
+
- Evidence: capture `007`; `version-analysis.json`;
|
|
468
|
+
`migration-checklist.json`.
|
|
469
|
+
|
|
470
|
+
#### M-02 — Failure recovery guidance is inconsistent
|
|
471
|
+
|
|
472
|
+
- Reproduction: run the missing-upstream, missing-configuration, and invalid-
|
|
473
|
+
lineage scenarios.
|
|
474
|
+
- Expected: stage name, plain-language cause, and a concrete safe next step.
|
|
475
|
+
- Actual: missing upstream input exposes a raw `ENOENT` path; missing
|
|
476
|
+
configuration names a variable; lineage failure gives no next command.
|
|
477
|
+
- Product impact: recovery depends on internal CLI knowledge.
|
|
478
|
+
- Evidence: captures `004`, `008`, and `009`.
|
|
479
|
+
|
|
480
|
+
#### M-03 — Duplicate occurrences are hard to distinguish in Markdown
|
|
481
|
+
|
|
482
|
+
- Reproduction: open the two `langchain-openai` sections in the VinGrade
|
|
483
|
+
Markdown report.
|
|
484
|
+
- Expected: occurrence labels that identify manifest, declaration, or context.
|
|
485
|
+
- Actual: JSON identity is correct, but duplicate headings omit declared
|
|
486
|
+
version, manifest occurrence, and context ID.
|
|
487
|
+
- Product impact: a reviewer can attach a result to the wrong declaration.
|
|
488
|
+
- Evidence: capture `007`; `repository-impact.md`;
|
|
489
|
+
`version-analysis.json`.
|
|
490
|
+
|
|
491
|
+
#### M-04 — The required review report is publishable by default
|
|
492
|
+
|
|
493
|
+
- Reproduction: run the post-report npm pack in capture `018`.
|
|
494
|
+
- Expected: internal review reports excluded unless they are intentional
|
|
495
|
+
public package documentation.
|
|
496
|
+
- Actual: the package includes `docs/**` except CLI-capture trees, so this
|
|
497
|
+
report is present in the 209-file tarball.
|
|
498
|
+
- Product impact: internal review conclusions can be published unintentionally.
|
|
499
|
+
- Evidence: captures `018` and `019`.
|
|
500
|
+
|
|
501
|
+
### Low
|
|
502
|
+
|
|
503
|
+
#### L-01 — Snapshot directory name becomes the repository name
|
|
504
|
+
|
|
505
|
+
- Reproduction: run discovery from the isolated `vingrade-online` snapshot.
|
|
506
|
+
- Expected: repository identity derived from project metadata or clearly
|
|
507
|
+
labeled as the directory name.
|
|
508
|
+
- Actual: the displayed repository name is `vingrade-online`, not VinGrade.
|
|
509
|
+
- Product impact: report identity is noisy in temporary and CI workspaces.
|
|
510
|
+
- Evidence: captures `005` and `007`.
|
|
511
|
+
|
|
512
|
+
#### L-02 — Help is command-first rather than outcome-first
|
|
513
|
+
|
|
514
|
+
- Reproduction: run the packaged `--help` smoke test.
|
|
515
|
+
- Expected: the primary upgrade decision, risk, and roadmap outcomes are
|
|
516
|
+
explained before diagnostic commands.
|
|
517
|
+
- Actual: commands and artifact contracts dominate the help output.
|
|
518
|
+
- Product impact: a first-time user must infer the intended product workflow.
|
|
519
|
+
- Evidence: capture `014`.
|
|
520
|
+
|
|
521
|
+
#### L-03 — Final summary omits the best next action
|
|
522
|
+
|
|
523
|
+
- Reproduction: inspect the online and empty-cache offline final summaries.
|
|
524
|
+
- Expected: name the unavailable or failed dependency and state the most useful
|
|
525
|
+
safe recovery step.
|
|
526
|
+
- Actual: summaries show aggregate counts and artifact paths only.
|
|
527
|
+
- Product impact: the developer must open JSON before knowing what to do next.
|
|
528
|
+
- Evidence: captures `003` and `007`.
|
|
529
|
+
|
|
530
|
+
### Future scope
|
|
531
|
+
|
|
532
|
+
- Cross-model comparison once a second valid model configuration exists.
|
|
533
|
+
- Explicit framework/core/small/internal/transitive classification.
|
|
534
|
+
- Historical trend presentation using the existing stable identities.
|
|
535
|
+
|
|
536
|
+
## 9. Over-engineering Assessment
|
|
537
|
+
|
|
538
|
+
### Complexity that should remain
|
|
539
|
+
|
|
540
|
+
- strict schemas and invariant validation;
|
|
541
|
+
- exact-byte lineage and canonical identities;
|
|
542
|
+
- atomic private artifact writers;
|
|
543
|
+
- occurrence-level identity;
|
|
544
|
+
- fail-closed provider and trust behavior;
|
|
545
|
+
- qualification integrity and exact runtime matching;
|
|
546
|
+
- deterministic progress and controlled cancellation;
|
|
547
|
+
- evidence freshness and cache state.
|
|
548
|
+
|
|
549
|
+
These controls directly prevent unsupported recommendations and corrupted
|
|
550
|
+
handoffs.
|
|
551
|
+
|
|
552
|
+
### Complexity that should be hidden from the default user
|
|
553
|
+
|
|
554
|
+
- qualification IDs and identity digests;
|
|
555
|
+
- internal reason codes when a plain-language next step exists;
|
|
556
|
+
- evaluation, scorecard, benchmark, conformance, and governance commands in
|
|
557
|
+
the primary help path;
|
|
558
|
+
- the eight-artifact implementation chain.
|
|
559
|
+
|
|
560
|
+
The default CLI should present decisions, reasons, risks, and next steps, then
|
|
561
|
+
link to diagnostic details.
|
|
562
|
+
|
|
563
|
+
### Complexity not yet proving product value
|
|
564
|
+
|
|
565
|
+
Migration Planning has extensive schemas, qualification policy, prompt
|
|
566
|
+
identity, evaluation criteria, trust validation, and artifact lineage. In the
|
|
567
|
+
representative workflow this complexity produced zero grounded actions and a
|
|
568
|
+
handoff that still required full repository investigation.
|
|
569
|
+
|
|
570
|
+
This does not justify removing the safety architecture. It means default
|
|
571
|
+
enablement and further abstraction should wait until the product can produce
|
|
572
|
+
a useful, evidence-bounded roadmap on representative repositories.
|
|
573
|
+
|
|
574
|
+
## 10. Release Recommendation
|
|
575
|
+
|
|
576
|
+
**`NO_GO`**
|
|
577
|
+
|
|
578
|
+
Automated regression, packaging, integrity, privacy, and provider
|
|
579
|
+
qualification gates pass. Those results establish engineering safety; they do
|
|
580
|
+
not establish product alignment.
|
|
581
|
+
|
|
582
|
+
For a v0.5.0 release positioned as full dependency-upgrade intelligence with
|
|
583
|
+
AI Migration Planning, the real workflow must first:
|
|
584
|
+
|
|
585
|
+
1. know the actual baseline;
|
|
586
|
+
2. state whether to upgrade, keep, or investigate;
|
|
587
|
+
3. avoid non-impact claims outside analyzer coverage;
|
|
588
|
+
4. produce an implementation-usable bounded roadmap; and
|
|
589
|
+
5. signal partial provider failure correctly.
|
|
590
|
+
|
|
591
|
+
Migration Checklist should remain experimental.
|
|
592
|
+
|
|
593
|
+
## 11. Next Tasks
|
|
594
|
+
|
|
595
|
+
1. Resolve exact current versions from supported lockfiles and environments,
|
|
596
|
+
then prove React and `remark-gfm` already-at-target decisions.
|
|
597
|
+
2. Add a deterministic decision layer and outcome-first console/Markdown
|
|
598
|
+
summary: upgrade now, plan, keep, insufficient evidence, or not analyzed.
|
|
599
|
+
3. Make repository-impact conclusions coverage-aware by ecosystem; never emit
|
|
600
|
+
dependency-not-used/not-impacted when the language has no analyzer.
|
|
601
|
+
4. Define and validate a bounded handoff roadmap containing evidence-supported
|
|
602
|
+
actions, areas, verification, approvals, and ordering; keep Migration
|
|
603
|
+
Checklist experimental until it passes a real-project handoff test.
|
|
604
|
+
5. Make partial provider/output failures explicit in exit semantics and CLI
|
|
605
|
+
recovery guidance, while retaining successful occurrence results.
|