@thomasminh1995/depverdict 0.6.0-alpha.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (252) hide show
  1. package/LICENSE +21 -0
  2. package/README.md +518 -0
  3. package/bin/depverdict.js +7 -0
  4. package/bin/upgradelens.js +7 -0
  5. package/docs/GR-01-Semantic-Grounding-Failure-Analysis.md +309 -0
  6. package/docs/GR-02-Versioned-Action-Evaluation-Criteria.md +187 -0
  7. package/docs/GR-03-Extractive-Contract-Safety-Experiment.md +242 -0
  8. package/docs/GR-04-Versioned-Production-Extractive-Contract.md +154 -0
  9. package/docs/IA-01-Repository-Usage-Discovery.md +122 -0
  10. package/docs/IA-02-Repository-Impact-Analysis.md +118 -0
  11. package/docs/IA-03-Repository-Impact-Evidence.md +160 -0
  12. package/docs/IA-04-CLI-Orchestration.md +235 -0
  13. package/docs/IA-05-Real-Provider-Validation.md +235 -0
  14. package/docs/IA-05-VinGrade-Validation.md +339 -0
  15. package/docs/MVP-01.md +48 -0
  16. package/docs/MVP-02-Architecture.md +536 -0
  17. package/docs/MVP-02-CLI-HTTP-Runtime.md +50 -0
  18. package/docs/MVP-02-HTTP-Lifecycle.md +37 -0
  19. package/docs/MVP-02-Knowledge-Manifest-Generation.md +75 -0
  20. package/docs/MVP-02-Knowledge-Manifest.md +279 -0
  21. package/docs/MVP-02-Knowledge-Research-Orchestration.md +62 -0
  22. package/docs/MVP-02-Knowledge-Store.md +87 -0
  23. package/docs/MVP-02-PyPI-Registry-Adapter.md +97 -0
  24. package/docs/MVP-02-Research-Planning.md +196 -0
  25. package/docs/MVP-02-Source-Provenance.md +81 -0
  26. package/docs/MVP-02-npm-Registry-Adapter.md +99 -0
  27. package/docs/RR-01-End-to-End-and-Real-Provider-Validation.md +393 -0
  28. package/docs/RR-01-RERUN-Extractive-Contract-Validation.md +566 -0
  29. package/docs/RR-02-Full-Product-Workflow-and-Developer-CLI-UX-Review.md +35 -0
  30. package/docs/RR02-FIX-01-Persistent-Qualification-Resolution.md +32 -0
  31. package/docs/RR02-FIX-02-Stage-aware-CLI-Progress-and-Heartbeat.md +28 -0
  32. package/docs/RR02-FIX-03-npm-Capture-Evidence-Exclusion.md +33 -0
  33. package/docs/RR02-FIX-03A-Complete-Package-Exclusion-and-Evidence-Commit.md +223 -0
  34. package/docs/RR02-FIX-04-Event-loop-safe-Heartbeat.md +393 -0
  35. package/docs/RR02-FIX-05-Materialize-Persisted-Qualification.md +207 -0
  36. package/docs/RR02-RERUN-CLI-Qualification-Progress-UX-and-Package-Validation.md +342 -0
  37. package/docs/VinGrade-MVP-02-Validation.md +480 -0
  38. package/docs/VinGrade-RC02-Live-Validation.md +273 -0
  39. package/docs/ai-capability-discovery.md +573 -0
  40. package/docs/ai-engineering-production-readiness.md +390 -0
  41. package/docs/ai-engineering-review.md +251 -0
  42. package/docs/ai-runtime-governance-discovery.md +754 -0
  43. package/docs/architecture-overview.md +78 -0
  44. package/docs/cli-progress.md +111 -0
  45. package/docs/decisions/diff-01-brand-distribution-identity.md +450 -0
  46. package/docs/decisions/diff-02-identity-compatibility-contract.md +274 -0
  47. package/docs/decisions/diff-03-repository-docs-community-migration.md +165 -0
  48. package/docs/decisions/diff-04-release-evidence-gap-acceptance.md +110 -0
  49. package/docs/discovery/mvp-05-ai-migration-planning-discovery.md +350 -0
  50. package/docs/gateway-runtime-discovery.md +614 -0
  51. package/docs/live-ai-validation.md +299 -0
  52. package/docs/migration-planning-qualification-resolution.md +99 -0
  53. package/docs/migrations/upgradelens-to-depverdict.md +97 -0
  54. package/docs/mp-r03-deterministic-upgrade-decision-architecture.md +135 -0
  55. package/docs/mp-r04-evidence-bounded-migration-handoff-architecture.md +116 -0
  56. package/docs/mp-r05-product-completion-and-decision-first-cli-architecture.md +162 -0
  57. package/docs/mvp-05-deterministic-context-runtime.md +127 -0
  58. package/docs/mvp-05-migration-checklist-contract.md +166 -0
  59. package/docs/mvp-05-migration-checklist-orchestration.md +105 -0
  60. package/docs/mvp-05-migration-evaluation-and-qualification.md +135 -0
  61. package/docs/mvp-05-provider-neutral-generator.md +79 -0
  62. package/docs/ollama-local-smoke-validation.md +156 -0
  63. package/docs/openai-compatible-runtime-discovery.md +789 -0
  64. package/docs/openrouter-one-dependency-validation.md +205 -0
  65. package/docs/oss-02-package-guard-hardening-architecture.md +131 -0
  66. package/docs/oss-04-public-ci-package-metadata-architecture.md +142 -0
  67. package/docs/package-content-policy.md +98 -0
  68. package/docs/releases/v0.5.0-technical-preview.md +114 -0
  69. package/docs/releases/v0.6.0-alpha.1-depverdict-preview.md +101 -0
  70. package/docs/reviews/diff-02-identity-contract-compatibility.md +413 -0
  71. package/docs/reviews/diff-03-repository-docs-community-migration.md +364 -0
  72. package/docs/reviews/diff-04-depverdict-distribution-identity-readiness-rereview.md +453 -0
  73. package/docs/reviews/diff-04-fix-post-rename-identity-release-remediation.md +310 -0
  74. package/docs/reviews/diff-05-final-preview-distribution-qualification.md +561 -0
  75. package/docs/reviews/mvp-05-final-product-value-workflow-rereview.md +471 -0
  76. package/docs/reviews/mvp-05-product-workflow-review.md +605 -0
  77. package/docs/reviews/oss-01-duplicate-artifact-investigation-cleanup.md +411 -0
  78. package/docs/reviews/oss-02-package-guard-hardening.md +303 -0
  79. package/docs/reviews/oss-03-community-scaffolding.md +378 -0
  80. package/docs/reviews/oss-04-public-ci-package-metadata.md +439 -0
  81. package/docs/reviews/oss-05-technical-preview-qualification.md +482 -0
  82. package/docs/reviews/upgradelens-vs-upgradedepdetective-source-comparison.md +355 -0
  83. package/docs/reviews/v0.5.0-pre-release-smoke.md +263 -0
  84. package/docs/reviews/v0.5.0-version-bump-release-verification.md +306 -0
  85. package/docs/runtime-contract-discovery.md +521 -0
  86. package/docs/structured-output-compatibility-report.md +100 -0
  87. package/docs/ts-fix-01-exact-duplicate-occurrence-target-selection-architecture.md +111 -0
  88. package/docs/version-analysis-architecture.md +827 -0
  89. package/eval/README.md +86 -0
  90. package/eval/datasets/generic/declared-constraint.json +59 -0
  91. package/eval/datasets/generic/evidence-conflict.json +73 -0
  92. package/eval/datasets/generic/major-breaking-release.json +66 -0
  93. package/eval/datasets/generic/missing-evidence.json +46 -0
  94. package/eval/datasets/generic/patch-release-low.json +59 -0
  95. package/eval/datasets/node/axios-patch-low.json +59 -0
  96. package/eval/datasets/node/react-major-breaking.json +66 -0
  97. package/eval/datasets/node/react-minor-compatibility.json +66 -0
  98. package/eval/datasets/python/fastapi-deprecation.json +66 -0
  99. package/eval/datasets/python/pydantic-major-breaking.json +66 -0
  100. package/eval/migration-planning/golden-dataset-v2.json +306 -0
  101. package/eval/migration-planning/golden-dataset.json +214 -0
  102. package/eval/schemas/expected-result.schema.json +88 -0
  103. package/eval/schemas/golden-case.schema.json +181 -0
  104. package/package.json +57 -0
  105. package/schemas/ai-scorecard.schema.json +132 -0
  106. package/schemas/benchmark-report.schema.json +189 -0
  107. package/schemas/benchmark.schema.json +55 -0
  108. package/schemas/capability-profile.schema.json +44 -0
  109. package/schemas/conformance-report.schema.json +150 -0
  110. package/schemas/deployment-profile.schema.json +64 -0
  111. package/schemas/evaluation-report.schema.json +158 -0
  112. package/schemas/knowledge-evidence-bundle.schema.json +150 -0
  113. package/schemas/knowledge-manifest.schema.json +548 -0
  114. package/schemas/metrics.schema.json +178 -0
  115. package/schemas/migration-checklist-extractive-candidate.schema.json +42 -0
  116. package/schemas/migration-checklist.schema.json +706 -0
  117. package/schemas/migration-evaluation-dataset-v2.schema.json +208 -0
  118. package/schemas/migration-evaluation-dataset.schema.json +204 -0
  119. package/schemas/migration-planning-qualification-record.schema.json +234 -0
  120. package/schemas/project-manifest.schema.json +308 -0
  121. package/schemas/qualification-record.schema.json +56 -0
  122. package/schemas/repository-impact-evidence.schema.json +232 -0
  123. package/schemas/repository-impact.schema.json +202 -0
  124. package/schemas/upgrade-decision.schema.json +273 -0
  125. package/schemas/usage-index.schema.json +179 -0
  126. package/schemas/version-analysis.schema.json +449 -0
  127. package/src/ai-runtime-debug.js +325 -0
  128. package/src/ai-runtime-error.js +42 -0
  129. package/src/ai-runtime.js +174 -0
  130. package/src/ai-scorecard.js +204 -0
  131. package/src/ai-version-analysis.js +484 -0
  132. package/src/artifact-root-compatibility.js +91 -0
  133. package/src/benchmark-report.js +111 -0
  134. package/src/benchmark-runner.js +191 -0
  135. package/src/canonical-json.js +69 -0
  136. package/src/cli.js +1299 -0
  137. package/src/conformance-report.js +158 -0
  138. package/src/conformance-runner.js +253 -0
  139. package/src/constants.js +73 -0
  140. package/src/cooperative-scheduler.js +79 -0
  141. package/src/dependencies.js +44 -0
  142. package/src/dependency-ai-context.js +625 -0
  143. package/src/detectors.js +253 -0
  144. package/src/discovery.js +234 -0
  145. package/src/ecosystem-version-adapter.js +294 -0
  146. package/src/environment-compatibility.js +77 -0
  147. package/src/evaluation-comparator.js +158 -0
  148. package/src/evaluation-report.js +76 -0
  149. package/src/evaluation-runner.js +248 -0
  150. package/src/evidence-source-adapter.js +472 -0
  151. package/src/files.js +89 -0
  152. package/src/governance-diagnostics.js +64 -0
  153. package/src/governance-loader.js +63 -0
  154. package/src/governance-metadata.js +346 -0
  155. package/src/governance-validator.js +360 -0
  156. package/src/http/bounded-fetch.js +278 -0
  157. package/src/http/cli-http-runtime.js +44 -0
  158. package/src/impact/input-loader.js +157 -0
  159. package/src/impact/matcher.js +40 -0
  160. package/src/impact/repository-impact.js +199 -0
  161. package/src/impact/runtime.js +24 -0
  162. package/src/impact/status.js +62 -0
  163. package/src/impact/writer.js +30 -0
  164. package/src/impact-evidence/input-loader.js +202 -0
  165. package/src/impact-evidence/repository-impact-evidence.js +234 -0
  166. package/src/impact-evidence/runtime.js +16 -0
  167. package/src/impact-evidence/writer.js +30 -0
  168. package/src/index.js +563 -0
  169. package/src/installed-version-baseline.js +196 -0
  170. package/src/knowledge-cache.js +324 -0
  171. package/src/knowledge-evidence-bundle.js +101 -0
  172. package/src/knowledge-evidence-producer.js +233 -0
  173. package/src/knowledge-manifest-builder.js +188 -0
  174. package/src/knowledge-manifest-writer.js +32 -0
  175. package/src/knowledge-manifest.js +255 -0
  176. package/src/knowledge-research.js +615 -0
  177. package/src/metrics-engine.js +205 -0
  178. package/src/migration-checklist/ai-candidate.js +320 -0
  179. package/src/migration-checklist/assembler.js +37 -0
  180. package/src/migration-checklist/context-runtime.js +828 -0
  181. package/src/migration-checklist/evaluation/action-criteria.js +244 -0
  182. package/src/migration-checklist/evaluation/comparator-v2.js +332 -0
  183. package/src/migration-checklist/evaluation/comparator.js +279 -0
  184. package/src/migration-checklist/evaluation/dataset-v2.js +227 -0
  185. package/src/migration-checklist/evaluation/dataset.js +336 -0
  186. package/src/migration-checklist/evaluation/extractive-fixtures-v2.js +148 -0
  187. package/src/migration-checklist/evaluation/metrics-v2.js +226 -0
  188. package/src/migration-checklist/evaluation/metrics.js +158 -0
  189. package/src/migration-checklist/evaluation/qualification-v2.js +321 -0
  190. package/src/migration-checklist/evaluation/qualification.js +239 -0
  191. package/src/migration-checklist/evaluation/runner-v2.js +294 -0
  192. package/src/migration-checklist/evaluation/runner.js +194 -0
  193. package/src/migration-checklist/evaluation/scorecard-v2.js +106 -0
  194. package/src/migration-checklist/evaluation/scorecard.js +86 -0
  195. package/src/migration-checklist/extractive-candidate.js +166 -0
  196. package/src/migration-checklist/extractive-prompt.js +62 -0
  197. package/src/migration-checklist/generator.js +702 -0
  198. package/src/migration-checklist/grounding-policy.js +117 -0
  199. package/src/migration-checklist/input-loader.js +613 -0
  200. package/src/migration-checklist/migration-checklist.js +635 -0
  201. package/src/migration-checklist/presentation.js +292 -0
  202. package/src/migration-checklist/progress.js +96 -0
  203. package/src/migration-checklist/prompt.js +83 -0
  204. package/src/migration-checklist/qualification-guard.js +462 -0
  205. package/src/migration-checklist/qualification-resolution.js +122 -0
  206. package/src/migration-checklist/qualification-store.js +225 -0
  207. package/src/migration-checklist/runtime.js +205 -0
  208. package/src/migration-checklist/verification.js +134 -0
  209. package/src/migration-checklist/writer.js +35 -0
  210. package/src/openai-compatible-provider.js +451 -0
  211. package/src/orchestration/failure-log.js +32 -0
  212. package/src/orchestration/pipeline.js +200 -0
  213. package/src/orchestration/progress-events.js +337 -0
  214. package/src/orchestration/progress-reporter.js +131 -0
  215. package/src/orchestration/text-writer.js +22 -0
  216. package/src/portable.js +13 -0
  217. package/src/product-completion.js +249 -0
  218. package/src/project-manifest-input.js +90 -0
  219. package/src/project-manifest.js +141 -0
  220. package/src/python-requirements.js +137 -0
  221. package/src/registry/npm-packument.js +256 -0
  222. package/src/registry/npm-registry-adapter.js +262 -0
  223. package/src/registry/pypi-project.js +300 -0
  224. package/src/registry/pypi-registry-adapter.js +235 -0
  225. package/src/registry/sanitize-registry-body.js +51 -0
  226. package/src/renderers/console.js +160 -0
  227. package/src/renderers/impact-presentation.js +278 -0
  228. package/src/renderers/markdown.js +172 -0
  229. package/src/research-plan.js +455 -0
  230. package/src/runtime-conformance.js +275 -0
  231. package/src/source-provenance.js +393 -0
  232. package/src/source-url.js +62 -0
  233. package/src/structured-output-schema.js +66 -0
  234. package/src/target-selector.js +306 -0
  235. package/src/upgrade-decision/input-loader.js +107 -0
  236. package/src/upgrade-decision/presentation.js +43 -0
  237. package/src/upgrade-decision/runtime.js +21 -0
  238. package/src/upgrade-decision/upgrade-decision.js +626 -0
  239. package/src/upgrade-decision/writer.js +30 -0
  240. package/src/usage/analyzer-registry.js +63 -0
  241. package/src/usage/coverage.js +116 -0
  242. package/src/usage/input-loader.js +139 -0
  243. package/src/usage/js/analyzer.js +240 -0
  244. package/src/usage/js/parser.js +21 -0
  245. package/src/usage/runtime.js +187 -0
  246. package/src/usage/scope.js +44 -0
  247. package/src/usage/source-files.js +50 -0
  248. package/src/usage/usage-index.js +217 -0
  249. package/src/usage/writer.js +31 -0
  250. package/src/version-analysis-loader.js +203 -0
  251. package/src/version-analysis-manifest.js +314 -0
  252. package/src/version-analysis-writer.js +30 -0
@@ -0,0 +1,105 @@
1
+ # MVP-05 Migration Checklist Orchestration
2
+
3
+ ## Scope
4
+
5
+ MP-05 closes the experimental application path from the validated UpgradeLens artifacts, including persisted Upgrade Decision, to `.upgradelens/migration-checklist.json`, console output, and the existing Markdown report. MP-R04 upgrades that artifact to the evidence-bounded Migration Handoff contract without broadening the extractive or qualification boundary. Migration Checklist remains disabled by default.
6
+
7
+ The command is explicit:
8
+
9
+ ```text
10
+ upgradelens analyze <repository> --experimental-migration-checklist
11
+ ```
12
+
13
+ Without the flag, the default analysis pipeline is unchanged, no migration qualification is required, and no Migration Checklist artifact is created.
14
+
15
+ ## Runtime flow
16
+
17
+ ```text
18
+ Seven legacy exact-byte artifacts + persisted Upgrade Decision
19
+
20
+ MP-02 context preparation and deterministic fallbacks
21
+
22
+ Task/provider/model/adapter qualification guard
23
+
24
+ extractive v2 provider-neutral generation and trust validation
25
+
26
+ MP-01 final builder, schema and invariant validation
27
+
28
+ Atomic migration-checklist.json write
29
+
30
+ Pure Migration Checklist presentation view model
31
+
32
+ Console summary + Repository Impact Markdown section
33
+ ```
34
+
35
+ The optional stage is inserted after Repository Impact Evidence and before Markdown Report. `ANALYSIS_STAGES` remains the default stage list; an opt-in stage list is created only for the experimental command.
36
+
37
+ ## Qualification policy
38
+
39
+ The guard binds qualification to `migration-planning.v2`, dataset v2 and evaluation identities, policy digest, prompt identity/digest, candidate schema digest, generator/trust/presentation identities, provider, model, runtime adapter, and runtime mode.
40
+
41
+ - A matching real-provider `QUALIFIED` or `QUALIFIED_WITH_LIMITATIONS` record is accepted for that exact identity.
42
+ - `NOT_QUALIFIED` always blocks generation.
43
+ - Missing, fake, insufficient, or mismatched qualification is never called qualified.
44
+ - Explicit experimental opt-in may proceed with visible artifact/report limitations.
45
+ - Missing provider/model/adapter metadata adds a separate limitation.
46
+
47
+ The conservative missing-record message is represented by `MIGRATION_PROVIDER_NOT_QUALIFIED`: the configured provider/model has not been qualified for `migration-planning.v2`, and every generated instruction requires human review. Version Analysis and historical v1 qualification are not consulted.
48
+
49
+ Real-provider migration generation remains disabled by default. Having an API key or endpoint does not enable this stage.
50
+
51
+ ## Assembly and artifact writer
52
+
53
+ The assembler accepts only normalized MP-02 preparation, normalized MP-03 generation, a qualification-guard result, and an injected timestamp. It verifies the MP-02/MP-03 lineage input is identical and calls the MP-01 `buildMigrationChecklist` and validator. It does not edit AI instructions or create actions.
54
+
55
+ The writer validates before touching the target, serializes with the MP-01 serializer, writes a private temporary file in the target directory, syncs it, and atomically renames it. A failed validation or write does not replace an existing artifact. The public result is the portable path `.upgradelens/migration-checklist.json`, not an absolute local path.
56
+
57
+ ## Failure model
58
+
59
+ Fatal artifact/schema/lineage/reference/programming/write errors fail the Migration Checklist stage and stop later stages. Provider errors, abstention, invalid candidate output, trust rejection, unsupported evidence, and unsupported usage coverage remain package-local deterministic fallbacks. One context cannot remove another context's safe result.
60
+
61
+ All AI-selected official-guidance items remain location-free and require human review. Candidate locations remain deterministic positive `impactEvidenceId + symbol + file` records.
62
+
63
+ ## Progress
64
+
65
+ Migration progress events are operational-only and bounded:
66
+
67
+ - `stage:start`, `stage:progress`, `stage:complete`, `stage:failed`;
68
+ - `migration:context-start`, `migration:context-complete`;
69
+ - `migration:abstained`, `migration:trust-rejected`, `migration:fallback`;
70
+ - `migration:artifact-written`.
71
+
72
+ Events may contain counts, package display name, constrained outcome/reason codes, qualification state, and portable artifact path. They never contain prompts, evidence content, model output, provider error text, credentials, source code, or absolute paths. Missing or failing listeners do not change generation semantics.
73
+
74
+ `--progress auto|interactive|plain` controls the experimental stage display. `auto` selects a simple symbol-based interactive view for TTY and stable line-oriented output for CI. Neither mode uses fake percentages; the implementation reports processed/total contexts. Plain mode uses no cursor control or animation.
75
+
76
+ ## Presentation
77
+
78
+ Console and Markdown use the same deterministic view model built solely from the validated final artifact. They do not reload evidence or recalculate eligibility.
79
+
80
+ Required wording distinguishes:
81
+
82
+ - “AI-selected official guidance” rendered deterministically from an accepted exact span;
83
+ - a “candidate review location” from a proven affected call site;
84
+ - unknown current versions from exact baselines;
85
+ - a `registryLatest` fact from a recommendation;
86
+ - checklist `COMPLETE` coverage from migration completion or upgrade safety.
87
+
88
+ Evidence is displayed as artifact evidence IDs because the current Migration Checklist contract contains IDs but no source URLs. Renderers do not invent links.
89
+
90
+ ## Provider neutrality and limitations
91
+
92
+ The stage accepts any injected `AiRuntime`. Provider configuration continues to use the existing generic/OpenAI-compatible CLI runtime boundary; MP-05 contains no OpenRouter-specific provider, model, endpoint, or key logic.
93
+
94
+ Current limitations remain visible:
95
+
96
+ - fake qualification is not real-provider qualification;
97
+ - exact excerpts do not prove semantic entailment;
98
+ - leading-dash flags and unsupported plain-language instructions expose known lexical/semantic gaps;
99
+ - Usage Analyzer coverage is global rather than per project and Python has no usage analyzer;
100
+ - verification extraction is currently limited to safe generic Node package
101
+ script roles and never executes them;
102
+ - no structured recovery instruction is currently available;
103
+ - progress UX is intentionally basic pending RR-02.
104
+
105
+ RR-01 should validate the complete artifact pipeline on representative repositories. RR-02 should review interactive wording, density, failure diagnostics, and developer ergonomics. Real-provider enablement requires separate provider portability validation and task-specific qualification.
@@ -0,0 +1,135 @@
1
+ # MVP-05 Migration Evaluation and Qualification
2
+
3
+ ## Why MP-04 exists
4
+
5
+ MP-03 proves that candidate parsing, exact excerpt checks, deterministic ownership, and fail-closed fallback work. It does not prove that a generated instruction is semantically supported by its excerpt, that a model abstains at the right time, or that a particular provider/model is suitable for `migration-planning.v1`.
6
+
7
+ MP-04 adds a task-specific quality gate before MP-05 writer/orchestration work. It does not generate the final artifact, scan a repository, or change MP-01 through MP-03 production semantics.
8
+
9
+ ## Evaluation layers
10
+
11
+ 1. **Deterministic policy layer** validates the dataset, candidate schema, evidence allowlist, exact excerpts, identifier and prohibited-capability guards, identity, locations, human review, and stable replay.
12
+ 2. **Recorded fake-runtime layer** runs MP-03 end to end with versioned candidates and failures. It is offline, deterministic, and is the default.
13
+ 3. **Real-provider layer** accepts an explicitly injected provider-neutral `AiRuntime`. It is never selected by default and is not part of `npm test` or `npm run check`.
14
+
15
+ The report keeps three views separate:
16
+
17
+ - raw candidate behavior, including unsupported attempts and abstention;
18
+ - trust acceptance/rejection and whole-candidate containment;
19
+ - final published checklist behavior after deterministic fallback.
20
+
21
+ A model hallucination that is rejected therefore hurts raw/provider quality while still passing the no-leak published-output gate.
22
+
23
+ ## Dataset and coverage
24
+
25
+ The strict dataset is `eval/migration-planning/golden-dataset.json`, schema version `1.0.0`, task `migration-planning.v1`. Its JSON Schema rejects additional properties and its loader rejects duplicate or unstable case IDs, unknown evidence refs, inconsistent outcomes, inconsistent locations, missing required ecosystem/sample coverage, and invalid policy-probe expectations.
26
+
27
+ The initial 10 cases cover Node.js/npm, Python/PyPI, and generic contexts. They include explicit publisher/official actions, multiple evidence and multiple actions, abstention for change-only/announcement/ambiguous evidence, invalid output, runtime failure, whole-candidate rejection, exact positive JS/TS locations, unknown current version, `registryLatest`, and unsupported Usage Index coverage. Sixteen compact adversarial probes cover invented identifiers/config/flags, wrong or paraphrased excerpts, cross-context refs, URL/code/diff/command, ordering/prerequisite, rollback, effort, confidence/safety, and model-owned repository locations.
28
+
29
+ Acceptable action concepts use deterministic required/forbidden tokens on controlled fixtures. This is an evaluation oracle for the fixture, not a general semantic-entailment proof and not an LLM judge.
30
+
31
+ ## Metrics and denominator semantics
32
+
33
+ Every rate stores `value`, `numerator`, and `denominator`. A zero denominator produces `value: null`; it never produces an artificial 100%.
34
+
35
+ Metric groups are:
36
+
37
+ - grounding: evidence-reference precision/coverage, exact excerpt rate, action-support precision;
38
+ - hallucination/safety: unsupported actions, invented identifiers/URLs, prohibited attempts, and published unsupported actions;
39
+ - abstention: precision, recall, false abstention, and missed abstention;
40
+ - trust containment: rejection precision, unsafe containment, safe acceptance, and whole-candidate rejection;
41
+ - policy preservation: locations, identity, human review, version uncertainty, eligibility, and deterministic replay;
42
+ - runtime: schema/generated/abstained/rejected/failed counts, expected versus unexpected failures, case isolation, and policy-probe counts.
43
+
44
+ Quality metrics exclude deliberate runtime-failure and oracle-unsafe fixtures. Those fixtures remain visible in runtime, containment, and gate results.
45
+
46
+ ## Critical gates
47
+
48
+ The following are evaluated independently from quality averages:
49
+
50
+ - valid published evidence refs and exact excerpts;
51
+ - no published unsupported or prohibited instruction;
52
+ - no AI-owned location;
53
+ - mandatory human review;
54
+ - preserved package/finding/version identity and uncertainty;
55
+ - `registryLatest` is not a recommendation;
56
+ - unsupported usage is not represented as unused, safe, or not impacted;
57
+ - eligibility preservation and deterministic post-processing.
58
+
59
+ Any critical gate or critical threshold failure yields `NOT_QUALIFIED`.
60
+
61
+ ## Versioned thresholds and verdicts
62
+
63
+ Policy `1.0.0` requires zero published unsupported actions; 100% unsafe containment, location/identity/human-review/version/eligibility preservation, and deterministic replay. Initial quality thresholds are evidence precision and exact excerpts at 0.98, action support at 0.90, safe acceptance at 0.80, and abstention precision/recall at 0.85. Minimum coverage is three actionable quality cases, three abstention quality cases, ten adversarial probes, and all three ecosystems.
64
+
65
+ Verdicts are:
66
+
67
+ - `QUALIFIED`: complete real-provider identity, sufficient coverage, all gates and thresholds pass, and no limitation;
68
+ - `QUALIFIED_WITH_LIMITATIONS`: no critical failure, but a non-critical quality limitation or fake-only evidence remains;
69
+ - `NOT_QUALIFIED`: a critical gate or safety threshold fails;
70
+ - `INSUFFICIENT_EVIDENCE`: coverage, runtime completion, or provider/model lineage is insufficient.
71
+
72
+ Fake-runtime evaluation can reach at most `QUALIFIED_WITH_LIMITATIONS`. It never qualifies a real provider.
73
+
74
+ ## Identity and invalidation
75
+
76
+ Qualification identity includes task, dataset ID/version/digest, policy version/digest, prompt version, candidate-schema digest, generator/trust source identity, runtime mode, provider, model, and adapter. Observed provider/model identity is recorded and a mismatch makes evidence insufficient. Timestamps are injected by the caller and are not part of the qualification ID. Secrets, raw candidate payloads, and raw provider errors are not stored.
77
+
78
+ A change in provider, model, adapter, task, dataset, policy, prompt, candidate schema, or generator/trust source identity produces a different qualification identity and requires re-evaluation.
79
+
80
+ ## API usage
81
+
82
+ Offline recorded evaluation:
83
+
84
+ ```js
85
+ const report = await runMigrationEvaluation({
86
+ generatedAt: '2026-07-16T00:00:00.000Z'
87
+ });
88
+ ```
89
+
90
+ Explicit real-provider evaluation:
91
+
92
+ ```js
93
+ const report = await runMigrationEvaluation({
94
+ mode: 'real',
95
+ runtime: configuredAiRuntime,
96
+ runtimeMetadata: {
97
+ provider: 'configured-provider',
98
+ model: 'configured-model',
99
+ adapter: 'configured-adapter'
100
+ },
101
+ generatedAt: outerClockTimestamp
102
+ });
103
+ ```
104
+
105
+ The outer caller owns provider configuration, retry/timeout behavior, and timestamp. MP-04 adds no provider SDK, environment lookup, command, writer, or network call of its own.
106
+
107
+ ## Current qualification and limitations
108
+
109
+ The versioned fake dataset passes every critical gate and quality threshold. The verdict is `QUALIFIED_WITH_LIMITATIONS` because it is a recorded fake runtime and because two deterministic probes expose known lexical/semantic gaps: a leading-dash flag form and a plain-language instruction can pass the current MP-03 lexical boundary despite not being supported by the fixture oracle.
110
+
111
+ These gaps are not published by the golden end-to-end candidates and do not create a critical leak in this run. They demonstrate why exact excerpts and lexical checks are not semantic proof. A real-provider qualification has not been run and real AI must not be enabled by default.
112
+
113
+ MP-05 writer/orchestration is implemented as an explicit experimental opt-in; see [`mvp-05-migration-checklist-orchestration.md`](./mvp-05-migration-checklist-orchestration.md). Default real-provider enablement remains blocked until the exact provider/model/runtime obtains a task-specific qualification with sufficient evidence.
114
+
115
+ ## Version 2 evaluation path
116
+
117
+ GR-02 adds an explicit, opt-in evaluation path without reinterpreting the historical `1.0.0` dataset or policy. The v1 loader, comparator, fake runtime, metrics, scorecard, and qualification identity remain available unchanged.
118
+
119
+ The v2 path uses:
120
+
121
+ - dataset `migration-planning-golden@2.0.0`;
122
+ - evaluation criteria `migration-action-evaluation@1.0.0`;
123
+ - comparator `2.0.0` and normalization `1.0.0`;
124
+ - qualification policy `2.0.0`;
125
+ - fixture roles `LIVE_QUALITY`, `RECORDED_CONTAINMENT`, and `INJECTED_FAILURE`.
126
+
127
+ Only live-quality cases can call an explicitly injected real runtime. Recorded containment candidates and injected failures always use deterministic local inputs. Action support and presentation specificity are measured independently: low specificity is a quality limitation, while unsupported or unmatched ambiguous published actions remain critical failures.
128
+
129
+ The v2 qualification identity includes the criteria ID/version/digest plus comparator and normalization versions. Dataset, criteria, normalization, policy, runtime, prompt, schema, or generator/trust identity changes therefore produce a different qualification ID.
130
+
131
+ The deterministic v2 run currently returns `NOT_QUALIFIED`, intentionally: the unchanged production trust validator accepts two recorded unsafe candidates already documented as known lexical/semantic gaps. The criteria now place those fixtures in the containment denominator, so the qualification boundary fails closed. The recorded safe `npm package` candidate is rejected by the unchanged command regex and is reported separately as a quality limitation, not as an unsafe leak.
132
+
133
+ That result is the historical free-form v1 baseline. GR-04 adds a separate `migration-planning.v2` extractive production identity. Its offline run contains all `17/17` recorded unsafe candidates, accepts the recorded safe candidate `1/1`, publishes no unsupported or ambiguous actions, passes every critical gate, and remains `QUALIFIED_WITH_LIMITATIONS` under fake mode. The production trust validator uses exact-span provenance and structural safety only; fixture action criteria remain evaluation-only. See [`GR-04-Versioned-Production-Extractive-Contract.md`](./GR-04-Versioned-Production-Extractive-Contract.md).
134
+
135
+ See [`GR-02-Versioned-Action-Evaluation-Criteria.md`](./GR-02-Versioned-Action-Evaluation-Criteria.md) for the full role, metric, compatibility, and requalification contract.
@@ -0,0 +1,79 @@
1
+ # MVP-05 Provider-Neutral Generator and Trust Validation
2
+
3
+ ## Scope and runtime boundary
4
+
5
+ MP-03 turns only MP-02 `eligibleContexts` into human-review migration checklist drafts. New experimental application runs use the extractive task identity `migration-planning.v2` and call the existing provider-neutral `AiRuntime.generateStructured()` contract once per eligible context. The historical free-form `migration-planning.v1` path remains only for evaluation reproducibility. The generator does not read artifacts or source files, access the network directly, scan a repository, write the final checklist, or invoke CLI/orchestration code.
6
+
7
+ The public multi-context API is `generateMigrationChecklistDrafts(prepared, { aiRuntime })`. The lower-level APIs expose prompt construction, candidate validation, trust validation, and one-context generation for MP-04 evaluation and MP-05 assembly.
8
+
9
+ ## Model-visible prompt boundary
10
+
11
+ The model receives a bounded projection containing:
12
+
13
+ - package/ecosystem/registry identity;
14
+ - declared, current, and target version facts, including baseline certainty and target policy;
15
+ - existing analysis result and breaking-finding identity;
16
+ - the bounded finding summary;
17
+ - selected official/publisher evidence content and exact allowlist;
18
+ - eligibility, location-coverage state, limitations, and mandatory human-review policy.
19
+
20
+ Source URLs, positive candidate locations, repository source/snippets, fallback records, unrelated evidence/dependencies, and provider configuration are excluded. Locations remain deterministic even when MP-02 has a positive usage match.
21
+
22
+ The prompt describes the model as an evidence transformation component, not an autonomous planner. It requires abstention when evidence lacks an explicit action and preserves two important uncertainties: an unknown current version remains unknown, and `registryLatest` is a registry fact rather than a recommendation. Unsupported usage coverage is never presented as unused, not impacted, or safe.
23
+
24
+ ## Candidate contract
25
+
26
+ The strict JSON Schema has no additional properties and contains only:
27
+
28
+ - `status`: `ACTIONABLE` or `ABSTAIN`;
29
+ - up to four bounded items, each with a bounded instruction, one to six unique selected evidence refs, and one exact bounded excerpt per ref;
30
+ - `abstentionReason`: null for `ACTIONABLE`, or one constrained reason for `ABSTAIN`.
31
+
32
+ In v2 the model emits only `status`, exact `(evidenceRef, actionExcerpt)` selections, and a constrained abstention reason. It cannot emit the final instruction, item IDs, package/finding/result identity, kind, basis, status/eligibility ownership, review state, URL, location, code, command, patch, prerequisite graph, ordering, rollback, effort, confidence, or completion/approval state.
33
+
34
+ ## Evidence and excerpt validation
35
+
36
+ Every evidence ref must exist in both the context allowlist and selected evidence. Each ref requires exactly one excerpt from that same record. Excerpt verification normalizes only line endings (`CRLF` and `CR` to `LF`) and then requires an exact substring; it does not trim, fold case, fuzzy match, search semantically, or accept a paraphrase.
37
+
38
+ An exact excerpt proves that the quoted text exists in selected evidence. It does not prove that the entire generated instruction is semantically entailed. This limitation is why every AI-authored item remains a draft requiring human review and why MP-04 must evaluate instruction-level evidence precision.
39
+
40
+ ## Trust policy
41
+
42
+ The validator reuses MP-01 instruction guards and adds small deterministic checks for obvious:
43
+
44
+ - URLs, Markdown code fences, code/diff/patch material, and shell/package-manager commands;
45
+ - rollback plans, effort estimates, numeric confidence, and dependency ordering/prerequisites;
46
+ - repository paths/files and safety/readiness/verification/completion claims;
47
+ - technical identifiers introduced as facts.
48
+
49
+ Backticked identifiers, flags, member names, common camel/snake-case API tokens, and call-form identifiers must occur exactly in a verified supporting excerpt. This conservative lexical check prevents common invented API/config claims; it is a guardrail, not semantic entailment or an NLP moderation framework.
50
+
51
+ MP-03 uses whole-candidate rejection. If any item fails trust validation, no item from that candidate is published. This avoids representing partially rejected model output as complete checklist coverage.
52
+
53
+ ## Deterministic ownership and construction
54
+
55
+ For a valid candidate, deterministic code supplies:
56
+
57
+ - `basis: AI_AUTHORED`;
58
+ - `kind: REVIEW_MIGRATION_INSTRUCTION`;
59
+ - the existing finding/package/version identity;
60
+ - stable item IDs and sorted evidence refs/items;
61
+ - `requiresHumanReview: true`;
62
+ - an empty AI-owned location list.
63
+
64
+ When MP-02 provides positive locations, MP-03 separately creates deterministic `REVIEW_CANDIDATE_USAGE` items. The model never receives or owns those locations. MP-02 limitations and fallback records are preserved and merged by analysis-result identity.
65
+
66
+ For identical normalized context and candidate output, parsing, trust checks, record construction, IDs, ordering, warnings, and summaries are deterministic. Model generation itself is not claimed to be deterministic. Core generation does not read the clock, and returned results are deeply immutable.
67
+
68
+ ## Abstention and failure behavior
69
+
70
+ A valid `ABSTAIN` candidate becomes a deterministic `NO_GROUNDED_ACTION` manual-review fallback with its constrained reason recorded as a limitation. It contains no AI-authored action.
71
+
72
+ Invalid JSON/schema/semantics, non-exact spans, trust rejection, and provider/runtime failure become `MANUAL_REVIEW_REQUIRED` deterministic fallbacks with constrained, sanitized warnings. Raw provider messages are not copied. A failure is package-local: other eligible contexts continue and all MP-02 fallback records remain available. Invalid context shape, duplicate identity, missing runtime, and programming invariants remain fatal. See [`GR-04-Versioned-Production-Extractive-Contract.md`](./GR-04-Versioned-Production-Extractive-Contract.md).
73
+
74
+ ## Deferred work
75
+
76
+ - MP-04 now provides migration-specific golden/adversarial evaluation, metrics, critical gates, a scorecard, and provider-specific qualification; see [`mvp-05-migration-evaluation-and-qualification.md`](./mvp-05-migration-evaluation-and-qualification.md). Qualification from Version Analysis is not inherited, and the fake runtime does not qualify a real provider.
77
+ - MP-05 implements final checklist assembly/writing, experimental pipeline/CLI orchestration, and presentation-only console/Markdown rendering; see [`mvp-05-migration-checklist-orchestration.md`](./mvp-05-migration-checklist-orchestration.md).
78
+
79
+ MP-03 adds no provider adapter, writer, renderer, source analyzer, code generator, patch/auto-fix, command generation, dependency graph, ordering, rollback, effort, confidence, or safety certification.
@@ -0,0 +1,156 @@
1
+ # LR-02 — Ollama Local Real Smoke Validation
2
+
3
+ Ngày validation: 2026-07-15
4
+
5
+ ## 1. Environment
6
+
7
+ | Item | Observed value |
8
+ | --- | --- |
9
+ | Host | macOS Darwin 25.5.0, arm64, Apple M5, 16 GiB unified memory |
10
+ | Local inference | Ollama 0.23.0, loopback only |
11
+ | UpgradeLens | 0.2.0, commit `27598df`, LR-01 working tree |
12
+ | Validation target | Current local VinGrade repository, commit `25811ef`; no repository path is persisted here |
13
+ | Date/time zone | 2026-07-15, Asia/Ho_Chi_Minh |
14
+
15
+ UpgradeLens was invoked against `.` from the VinGrade working repository. That repository was already dirty before LR-02. LR-02 did not mutate VinGrade source; generated `.upgradelens` artifacts remained untracked. No absolute target path was added to source or this report.
16
+
17
+ The Ollama service was not initially running. A temporary local service was started on `127.0.0.1:11434` without changing persistent system configuration, then stopped after validation; the final health check confirmed the port was no longer accepting connections.
18
+
19
+ ## 2. Local Model Inventory
20
+
21
+ Sanitized output from `ollama list`:
22
+
23
+ | Model | Displayed ID | Size | Modified |
24
+ | --- | --- | ---: | --- |
25
+ | `nomic-embed-text:latest` | `0a109f422b47` | 274 MB | 9 days ago |
26
+ | `qwen2.5-coder:7b` | `dae161e27b0e` | 4.7 GB | 9 days ago |
27
+ | `qwen3:latest` | `500a1f067a9f` | 5.2 GB | 7 weeks ago |
28
+ | `llama3:latest` | `365c0bd3c000` | 4.7 GB | 2 months ago |
29
+
30
+ `/api/version` reported `0.23.0`; `/v1/models` exposed the same four exact model tags. No model was pulled, downloaded, deleted, or modified.
31
+
32
+ ## 3. Selected Model
33
+
34
+ **Primary:** `qwen3:latest`, selected because the installed artifact is a Qwen3 8B Q4_K_M instruction/reasoning model and is the strongest match for structured reasoning under the stated selection policy.
35
+
36
+ **Secondary:** `llama3:latest`, an installed Meta-Llama-3-8B-Instruct Q4_0 artifact from a different family. It was invoked once on the same LangSmith context only after every primary analysis call timed out.
37
+
38
+ Both `latest` tags are mutable. This is smoke validation only and is not benchmark-reproducible. The displayed Ollama IDs above identify the artifacts observed during this run but do not create a qualification record.
39
+
40
+ ## 4. Protocol Probe
41
+
42
+ The direct probe used `POST http://localhost:11434/v1/chat/completions`, `qwen3:latest`, two simple non-VinGrade messages, `stream:false`, and a small strict JSON Schema containing `required`, `enum`, and `additionalProperties:false`.
43
+
44
+ Observed retained response:
45
+
46
+ | Check | Result |
47
+ | --- | --- |
48
+ | HTTP status | 200 |
49
+ | Actual model | `qwen3:latest` |
50
+ | Choice count | 1 |
51
+ | Finish reason | `stop` |
52
+ | Content | Valid JSON with the two schema-required enum values |
53
+ | Usage | 245 input, 14 output, 259 total tokens |
54
+ | Wall time | 78.7 seconds including local model work |
55
+
56
+ Ollama accepted native `response_format.type=json_schema`; the response matched the Chat Completions envelope and parsed to the required object.
57
+
58
+ An execution-session handoff caused the same harmless direct probe to be submitted twice before the first process was observed. Ollama recorded both as HTTP 200 (approximately 131 and 79 seconds). Neither request contained VinGrade data. This was an operator/tooling duplication, not runtime retry or model fallback; only the sanitized retained response above was used for the conformance conclusion.
59
+
60
+ ## 5. Real Artifact Summary
61
+
62
+ The production CLI first regenerated artifacts on the real VinGrade repository:
63
+
64
+ | Artifact | Summary |
65
+ | --- | --- |
66
+ | Project Manifest | 2 projects: one Node and one Python; 47 declarations, 46 unique packages |
67
+ | Knowledge Manifest | 46 researched; 45 resolved; 1 unavailable; 184 sources; 1 warning |
68
+ | Evidence Bundle | 45 evidence items for 45 packages; 2 warnings (`EVIDENCE_MISSING`, `REGISTRY_RESPONSE_INVALID`) |
69
+
70
+ The full repository contained 47 analysis contexts and 44 eligible model calls. Because CLI dependency filtering is not available, running all 44 calls would violate the bounded-smoke policy.
71
+
72
+ A temporary copy outside both repositories was therefore created with three declarations copied from the real VinGrade manifests: `axios`, `react`, and `langsmith`. Production `discover` and online `research` were rerun there. The bounded artifacts contained 2 projects, 3 resolved packages, 12 sources, 3 fresh registry evidence items, and no warnings. No fixture evidence was used, and no generated Knowledge Manifest or Evidence Bundle was edited manually.
73
+
74
+ ## 6. Pipeline Status
75
+
76
+ | Stage | Status | Evidence |
77
+ | --- | --- | --- |
78
+ | Real VinGrade `discover` | PASS | 2 project manifest validated |
79
+ | Real VinGrade `research` | PASS | 46 packages researched; manifest and bundle validated |
80
+ | Bounded subset `discover` | PASS | 2 projects / 3 declarations |
81
+ | Bounded subset `research` | PASS | 3/3 packages resolved with real registry evidence |
82
+ | Ollama protocol probe | PASS | Native JSON Schema request returned usable Chat Completions JSON |
83
+ | Primary `analyze-version` | PARTIAL | 3 real calls reached Ollama; all hit the 60-second runtime deadline |
84
+ | Secondary one-context analysis | PARTIAL | One Llama 3 call on the same LangSmith context hit the same deadline |
85
+ | Manifest validation | PASS | Primary and diagnostic secondary manifests pass schema and runtime invariants |
86
+
87
+ The exact candidate schema, prompt version 1, local JSON/Ajv path, and existing Trust Layer were not changed or bypassed.
88
+
89
+ ## 7. Runtime Result
90
+
91
+ Primary CLI configuration was `openai-compatible` with the loopback Chat Completions endpoint, exact model `qwen3:latest`, and no Authorization. The bounded batch expected and made exactly three calls. There was no retry or fallback.
92
+
93
+ | Runtime | Calls | Analyzed | Skipped | Failed | Result |
94
+ | --- | ---: | ---: | ---: | ---: | --- |
95
+ | `qwen3:latest` primary CLI batch | 3 | 0 | 0 | 3 | All `TIMEOUT` at the 60-second adapter deadline |
96
+ | `llama3:latest` secondary diagnostic | 1 | 0 | 0 | 1 | `TIMEOUT` after 60.1 seconds |
97
+
98
+ The primary manifest correctly recorded `failedCount:3`, risk `unknown`, `requiresHumanReview:true`, and package-local `TIMEOUT` limitations. The CLI still completed and wrote a valid manifest, proving that provider failure remained package-local.
99
+
100
+ Per-analysis token usage was unavailable because no analysis request completed. The successful protocol probe reported 259 total tokens. Runtime identity is recorded in this report from exact configuration, `/v1/models`, and Ollama server observations; it is intentionally absent from the portable Version Analysis Manifest.
101
+
102
+ ## 8. AI Output Review
103
+
104
+ No full Version Analysis candidate completed before the deadline, so semantic quality is not assessable.
105
+
106
+ | Dependency | Declared/current → target | Mode / evidence | Published result | Review |
107
+ | --- | --- | --- | --- | --- |
108
+ | `npm:axios` | `^1.17.0` / unresolved → `1.18.1` | `declaredConstraint`; `registryFact` | Failed, unknown risk, human review, zero refs/findings | NOT_ASSESSABLE |
109
+ | `npm:react` | `^19.2.6` / unresolved → `19.2.7` | `declaredConstraint`; `registryFact` | Failed, unknown risk, human review, zero refs/findings | NOT_ASSESSABLE |
110
+ | `pypi:langsmith` | `0.8.9` → `0.10.5` | `exactBaseline`; `registryFact` | Failed, unknown risk, human review, zero refs/findings | NOT_ASSESSABLE |
111
+
112
+ The runtime requested the exact candidate schema, but no candidate reached JSON parsing or Ajv. Consequently Trust Layer had no claims to accept or drop. Fail-closed results contained no invented evidence ID, invented URL, impact analysis, migration plan, or confident-low claim. Manifest evidence-reference invariants passed.
113
+
114
+ ## 9. Issues Found
115
+
116
+ | Classification | Finding |
117
+ | --- | --- |
118
+ | Local hardware/performance | Both installed 8B models exceeded the fixed 60-second analysis deadline on the full rendered prompt; Qwen3's tiny cold/warm probes also took longer than 60 seconds. |
119
+ | Configuration limitation | LR-01 has a bounded default timeout but no CLI/deployment-profile control for a qualified local timeout. This prevented technical completion without indicating an adapter protocol failure. |
120
+ | Evidence quality | The three bounded contexts contained only `registryFact`; even a completed result would require cautious interpretation and human review, especially for declared constraints. |
121
+ | Model capability | Qwen3 proved native JSON Schema on a tiny probe, but neither model demonstrated full UpgradeLens candidate completion within the allowed deadline. |
122
+ | Adapter bug | None demonstrated. Request mapping, timeout typing, package-local failure, sanitization, and manifest handling behaved as designed. |
123
+ | Operator tooling | Direct probe was duplicated once during command-session handoff; no private data was involved. |
124
+
125
+ No LR-01 code change was justified in this task. Changing prompt, schema, trust behavior, or silently weakening structured output would invalidate the smoke.
126
+
127
+ ## 10. Decision
128
+
129
+ ```text
130
+ TECHNICAL_SMOKE_PARTIAL
131
+ ```
132
+
133
+ Proven: local Ollama discovery, exact model selection, OpenAI-compatible Chat Completions transport, native JSON Schema acceptance, real VinGrade discovery/research, real registry evidence, real local model invocation, typed timeout isolation, fail-closed artifacts, and manifest validation.
134
+
135
+ Not proven: an eligible dependency completing output → JSON parse → exact Ajv → Trust Layer → `status=analyzed`. Therefore this run is not a technical smoke pass and provides no model-quality recommendation.
136
+
137
+ Security/privacy confirmation: endpoint was loopback; Authorization was absent; no API key, cloud inference, gateway transform, auto-download, prompt/evidence logging, source mutation, or artifact commit occurred. Generated private artifacts remained in the VinGrade untracked area or a temporary directory outside repositories.
138
+
139
+ ## 11. Qualification Status
140
+
141
+ ```text
142
+ Certification: EXPERIMENTAL
143
+ ```
144
+
145
+ This result is neither `SUPPORTED` nor `CERTIFIED`. It is not a benchmark, production qualification, model recommendation, or evidence that either model is suitable for MVP-04/MVP-05. Capability conformance and Golden Dataset evaluation remain incomplete.
146
+
147
+ ## 12. Recommended Next Step
148
+
149
+ Perform **model/runtime timeout compatibility hardening** before LR-G02:
150
+
151
+ 1. define an explicit, bounded local Deployment Profile timeout/output budget rather than changing the prompt or schema;
152
+ 2. add offline conformance coverage for timeout configuration and preservation of the exact model/schema tuple;
153
+ 3. rerun exactly one LangSmith context with `qwen3:latest` under the approved bounded profile;
154
+ 4. only after one candidate reaches Ajv and Trust Layer, proceed to **LR-G02 — Offline Conformance Suite**.
155
+
156
+ Do not compensate with weaker JSON mode, prompt-only JSON, model fallback, or a larger unbounded batch.