@dailephd/my-frontend-observer 0.8.1 → 0.9.0

package/docs/reports/v0.9-batch6-reference-materialization.md

@@ -0,0 +1,514 @@
+# v0.9 Batch 6 Report: Reference Materialization
+
+## 1. VERDICT
+
+PASS_V0_9_BATCH6_REFERENCE_MATERIALIZATION
+
+## 2. Repository identity
+
+1. Repository: `C:\Users\daile\Projects\my-frontend-observer`
+2. Branch: `master`
+3. Starting HEAD: `44415bc652f73e44bbfd5c457edf7f0f7daf2b5d`
+4. Ending HEAD before the final commit: `44415bc652f73e44bbfd5c457edf7f0f7daf2b5d`
+5. Package version: `0.8.1` (unchanged in `package.json` and `package-lock.json`)
+6. Viewer protocol version: `1.3.0` (was `1.2.0`)
+
+## 3. Prompt 5 entry gate
+
+1. Prompt 5 verdict: `PASS_V0_9_BATCH5_RUNTIME_INTENT_CONTRACT_PROMOTION`
+2. Prompt 5 commit: `44415bc652f73e44bbfd5c457edf7f0f7daf2b5d`
+3. Viewer protocol at entry: `1.2.0`
+4. Annotation schema: `1.0.0` (unchanged)
+5. External-reference schema: `1.0.0` (unchanged)
+6. Preflight found a clean tracked worktree on `master` at the required HEAD.
+7. `git ls-files -v docs/reports/v0.9-architecture-retrieval.md` printed `S`
+   at entry and before commit. The `skip-worktree` state was preserved. The
+   file was not restored, overwritten, deleted, or committed. No pull,
+   rebase, reset, stash, or clean was run.
+
+## 4. Materializable intent boundary
+
+1. Only confirmed `reference-region` (`create` or `refine`) and confirmed
+   `reference-requirement` items can be materialized.
+2. `referenceMaterializationBlockReason` in the new application service is
+   the single server rule:
+   1. uninterpreted items are refused;
+   2. candidate items are refused;
+   3. `inspect` and `asset-sensitive` are refused as informational intent;
+   4. runtime `change` intent is refused as not external-reference intent.
+3. Only the explicitly selected item ids are used. Unselected items never
+   contribute, even when they are confirmed.
+4. Selection is atomic. An empty, oversized (more than 100), duplicate, or
+   unknown selection, or any blocked item, rejects the whole request before
+   any write.
+5. The annotation must be saved. The API takes a saved annotation handle only,
+   and the UI disables materialization while the draft is unsaved or dirty.
+
+## 5. Source reference validation
+
+The service `materializeVisualAnnotationReference` checks, in order:
+
+1. The annotation passes the canonical `isValidVisualAnnotationArtifact`.
+2. `annotation.source.kind` is `external-reference`. A runtime annotation is
+   `invalid-source`.
+3. The supplied source passes the canonical
+   `isValidExternalReferenceArtifact`.
+4. The source exactly matches the annotation source identity:
+   `referenceId`, `referenceRequestId`, schema version, lifecycle, image owner
+   id, image SHA-256, and coordinate width and height. For an approved source
+   the image fields come from `sourceReference.image`. A mismatch is
+   `source-mismatch`.
+5. The supplied `sourceReferenceRoot` is read through the canonical
+   `readExternalReferenceArtifact` and must hold the same `referenceId`.
+   This keeps canonical supersession tied to the verified source.
+
+## 6. Source image resolution and verification
+
+1. The route never builds an image path from browser data or from the
+   annotation's image owner id.
+2. The route reads image bytes only through the existing safe
+   `resolveMedia` boundary:
+   1. `image` role for an imported source;
+   2. `source-image` role for an approved source (the owning imported image).
+3. The source artifact root comes from the source's own viewer handle through
+   `decodeArtifactHandle` and `resolveContainedDir`.
+4. The service recomputes SHA-256 over the supplied bytes and requires it to
+   equal the source image SHA-256.
+5. The bytes are passed unchanged to `importExternalReference`. Nothing is
+   transcoded, recompressed, or rasterized. Annotation marks are never baked
+   into the image.
+
+## 7. Region composition
+
+1. Final regions start from a copy of `sourceReference.regions ?? []`. The
+   source array and its region objects are never mutated.
+2. Selected refinements are applied first, in place.
+3. Selected creates are then appended in selection order.
+4. The final set is validated with the canonical
+   `isValidReferenceRegions(finalRegions, image.width, image.height)`. A
+   failure is `invalid-materialized-reference` and nothing is written.
+
+## 8. Region create behavior
+
+1. The confirmed `intent.region` is used exactly: id casing and rectangle are
+   kept, and no geometry is inferred from the mark.
+2. A create id that case-insensitively matches an existing source region is
+   rejected.
+3. Two selected creates with case-insensitively equal ids are rejected.
+4. Nothing is renamed.
+
+## 9. Region refine behavior
+
+1. A refine must target a region that exists in the selected source
+   reference. Ids match case-insensitively, as canonical region ids do.
+2. The refined region keeps its source id spelling and its position. Only the
+   rectangle changes. Unrelated regions keep their order and geometry.
+3. Two selected refinements of the same source region are rejected. Item
+   order never chooses a winner.
+4. Create then refine of the same new region in one request is rejected,
+   because the refine target is not a source region.
+
+## 10. Requirement composition
+
+1. Requirements are composed after regions, so a selected requirement may use
+   a region created in the same request.
+2. Each selected `intent.requirement` is validated with the canonical
+   `isValidRawReferenceRequirement` and is not reinterpreted.
+3. A requirement that references a region that is not in the final set (for
+   example an unselected create) is rejected by the canonical
+   `isValidReferenceRequirements`.
+4. A selected `region-relationship` requirement must hold for the final
+   geometry. The service checks it through the canonical
+   `deriveReferenceRequirementExpectation`. A contradicted relationship is
+   `invalid-materialized-reference`. This was needed because
+   `importExternalReference` validates relationship subjects structurally but
+   not against geometry.
+5. `region-measurement` requirements persist only the raw subject and
+   tolerance. No measured value is stored.
+6. Duplicates and conflicts are left to the canonical
+   `buildReferenceRequirement` and `isValidReferenceRequirements`. Nothing is
+   silently dropped.
+
+## 11. Existing requirement preservation
+
+1. Source requirements stay first, in their existing order.
+2. A private `toRawReferenceRequirement` copies only `category`,
+   `expectedDependentMode`, `subject`, and `tolerance`. `requirementId` is
+   never copied.
+3. The canonical import recomputes the ids. A unit test proves the ids of
+   unchanged source requirements are identical in the new reference.
+4. Selected new requirements are appended in `selectedItemIds` order.
+
+## 12. Applicability and label preservation
+
+1. `sourceReference.applicability` is passed through exactly when present and
+   omitted when absent. The annotation never changes viewport, theme,
+   application state, or authenticated state.
+2. `sourceReference.provenance.label` is passed through when present. When the
+   source has no label, none is invented. Viewer alias text is never used.
+
+## 13. Canonical external-reference import
+
+1. The new reference is persisted by exactly one call to the canonical
+   `importExternalReference(imageBytes, { outputLocation, cwd, label,
+   supersedesReferenceRoot, regions, requirements, applicability })`.
+2. `referenceRequestId` and `referenceId` are exactly the canonical service
+   output. Neither is constructed by Observer code outside that service.
+3. Import diagnostics `invalid-reference-region`,
+   `invalid-reference-requirement`, and `invalid-reference-applicability` map
+   to `invalid-materialized-reference`. Any other failure maps to
+   `persistence-failure`.
+4. `importExternalReference` and the external-reference writer were not
+   changed. No `BLOCKED_V0_9_BATCH6_CANONICAL_REFERENCE_SERVICE_CONFLICT` was
+   needed.
+5. New project path helpers in `src/projectWorkflow/projectPaths.ts`:
+   `projectReferencesRoot(projectRoot)` and `referenceOutputLocation()`
+   (`.frontend-observer/evidence/references`). Existing references are not
+   migrated.
+
+## 14. Supersession and lifecycle
+
+1. `supersedesReferenceRoot` is the verified source root, so the canonical
+   import sets `supersedesReferenceId` to the selected source `referenceId`.
+2. For an approved source this is the approved reference id, not the imported
+   image owner id. Unit and browser tests prove it.
+3. The new lifecycle is always `imported`, even from an approved source.
+4. Identical inputs give the same canonical `referenceRequestId` and a fresh
+   distinct `referenceId` (unit-tested).
+5. Source manifests and images are byte-identical after materialization. For
+   an approved source both the approved directory and the owning imported
+   directory were snapshotted.
+
+## 15. Materialization API
+
+1. Route: `POST /api/annotations/:handle/materialize-reference`, in
+   `src/viewerServer/httpServer.ts`. The use case lives in the new
+   `src/viewerServer/annotationReferenceMaterialization.ts` so the HTTP layer
+   stays thin.
+2. It uses the shared Prompt 2/5 gate `readAuthoringJsonRequest` (Host,
+   Origin, token, JSON content type, content encoding, body limit) and the
+   shared session write queue `runSerializedAuthoringWrite`. No second parser
+   or queue exists.
+3. The body is exactly `{ itemIds }`. The `itemIds` rule is now one shared
+   helper, `parseAuthoringItemIds` in `annotationAuthoring.ts`: a non-empty
+   array of at most 100 unique non-empty strings. The promotion parser uses
+   the same helper, with identical behavior and messages. Unknown fields
+   return 400.
+4. Resolution:
+   1. `getAnnotationView` resolves the annotation and its exact source handle.
+   2. The source must be `available` with family
+      `external-reference-imported` or `external-reference-approved`.
+   3. `loadArtifactByHandle` loads the source artifact.
+   4. Image bytes and the source root are resolved as in section 6.
+   5. The canonical service is called once with `referenceOutputLocation()`
+      and `cwd` set to the authoring project root.
+5. Success returns 201 with exactly `ok`, `referenceId`, `referenceRequestId`,
+   `handle` (`external-reference-imported:references%2F<referenceId>`),
+   `lifecycle: 'imported'`, `supersedesReferenceId`, `regionCount`,
+   `requirementCount`, and `approvalRequired: true`. It contains no paths, no
+   project root, and no token.
+6. Failure status mapping:
+   1. 400: malformed handle, invalid JSON, or invalid body shape.
+   2. 403: authoring or security failure, including a read-only viewer.
+   3. 404: unknown annotation, source reference unavailable, or source image
+      unavailable.
+   4. 409: handle is not a visual annotation, runtime annotation, invalid
+      source, or source mismatch.
+   5. 413: body too large.
+   6. 415: wrong content type or content encoding.
+   7. 422: unsupported or unconfirmed selection, invalid region or requirement
+      composition, stale relationship, or create/refine conflict.
+   8. 500: image read failure or persistence failure.
+   9. 405 with `Allow: POST` for GET and HEAD on the route. PUT, PATCH, and
+      DELETE keep the existing global 405.
+7. `VIEWER_PROTOCOL_VERSION` is `1.3.0`. The allowed POST surface is exactly
+   `POST /api/annotations`, `POST /api/annotations/:handle/promote-contract`,
+   and `POST /api/annotations/:handle/materialize-reference`.
+8. `materializeVisualAnnotationReference` and its options, result, and failure
+   code types are exported from `src/index.ts`.
+
+## 16. Viewer materialization UI
+
+1. `ReferenceAnnotationPanel` has a new "Confirmed reference intent" section,
+   shown only when authoring is available.
+2. It lists reference-region, reference-requirement, inspect, and
+   asset-sensitive items.
+   1. Only confirmed region and requirement items have enabled checkboxes.
+   2. Informational items are disabled with "Informational annotation intent
+      is not materialized into the external-reference contract."
+   3. Candidate items are disabled with "Candidate intent must be confirmed
+      before materialization."
+3. While the draft is unsaved or dirty, the section shows "Save the annotation
+   before materializing reference intent." and the button stays disabled.
+4. The button is "Materialize selected into new reference revision".
+5. The warning is always shown: "This creates a new imported reference
+   revision. The current reference remains unchanged. The new reference is
+   not approved automatically."
+6. Success shows the new reference id, lifecycle imported, supersedes,
+   region and requirement counts, approval required, and "The new reference
+   is imported but not approved. Use the existing explicit approval workflow
+   when ready."
+7. The selection resets after success and when the loaded annotation changes.
+8. The viewed reference is never switched. The new reference is discoverable
+   through the normal evidence index.
+9. The new hook `viewer/src/hooks/useAnnotationReferenceMaterialization.ts`
+   sends the in-memory token only in the authoring header, uses
+   `cache: 'no-store'`, never retries, and maps each failure status to a
+   bounded message.
+10. The presentation helpers `referenceMaterializationStatus` and
+    `isReferenceMaterializationListItem` were added to
+    `viewer/src/annotation/referenceIntent.ts`. The server remains
+    authoritative.
+11. There is no approval button and no project reference activation.
+
+## 17. No-approval/no-config-mutation guarantees
+
+1. The materialization path never calls `approveExternalReference`.
+2. Unit, route, and browser tests prove the reference directory gains exactly
+   one new imported artifact and no approved sibling.
+3. The route never reads or writes the project config. Route and browser tests
+   configure `acceptance.reference.approvedArtifact` and prove
+   `frontend-observer.json` is byte-identical after materialization.
+4. `src/projectWorkflow/projectConfig.ts` and its schema are unchanged.
+5. A unit test proves the new imported reference is accepted by the existing
+   explicit `approveExternalReference` workflow when run separately.
+
+## 18. Existing relationship and adequacy behavior
+
+1. A unit test runs the canonical `deriveReferenceRegionRelationships` on the
+   new reference and finds the normal `content follows-vertically header`
+   relationship.
+2. The same test runs the canonical `deriveReferenceRequirementAdequacy` and
+   gets the normal `adequate` result over all requirements.
+3. No annotation-specific relationship, adequacy, fidelity, or contract result
+   was added.
+
+## 19. Unit tests
+
+1. New `tests/unit/visualAnnotationReferenceMaterialization.test.ts` (17
+   tests):
+   1. Imported source create: regions appended after the source, imported
+      lifecycle, supersession, canonical identity, the canonical import called
+      exactly once, and source bytes unchanged.
+   2. Refine in place with order kept and the source rectangle untouched.
+   3. Case-insensitive refine that keeps the source id spelling.
+   4. Create plus requirement succeeds. The requirement alone (unselected
+      create) is rejected with no write.
+   5. Existing requirements stay first with identical recomputed ids, and new
+      requirements follow selection order.
+   6. Selected-only composition.
+   7. Applicability, label, image SHA, format, dimensions, and exact bytes are
+      preserved.
+   8. No invented label or applicability.
+   9. Approved source: supersedes the approved id, imported lifecycle, owning
+      image, both source directories unchanged, and no approved sibling.
+   10. The result is accepted by the existing approval workflow.
+   11. Same request identity with a fresh reference id.
+   12. Canonical relationship and adequacy output.
+   13. Rejections with no import call and no write: candidate, informational
+       (inspect and asset-sensitive), uninterpreted, unknown, empty,
+       duplicate, and mixed selections.
+   14. Create collisions, duplicate refine, and create then refine.
+   15. Stale `left-of` relationship after refine is rejected. The same
+       relationship with unchanged geometry succeeds.
+   16. A duplicate of an existing source requirement is rejected.
+   17. Runtime annotation, mismatched source, mismatched root, and wrong image
+       bytes.
+2. New `tests/unit/viewerAnnotationReferenceMaterialization.test.ts` (9
+   tests):
+   1. Closed body parser rules, including the 100-id bound.
+   2. Shared gate: bad token, Origin, Host 403, content type and content
+      encoding 415, shape and invalid JSON 400, oversized body 413, method
+      405. The service is never called and nothing is written.
+   3. Read-only viewer 403.
+   4. Unknown 404, malformed handle 400, wrong family 409, runtime annotation
+      409.
+   5. Source image moved away 404, approved source moved away 404.
+   6. Candidate, informational, unknown, and uncomposable selections 422 with
+      no write.
+   7. Selected-only success with an exact bounded key set, no paths or token,
+      a loadable `external-reference-imported` handle, imported lifecycle, no
+      approved sibling, and byte-identical project config with
+      `acceptance.reference` configured.
+   8. Approved source supersession and exact image bytes.
+   9. Shared write queue: while a materialization is held inside the queue, a
+      concurrent annotation save and a contract promotion do not complete.
+      Both complete only after it is released.
+3. Updated `tests/unit/viewerAuthoringSecurity.test.ts`: protocol `1.3.0`
+   (two assertions), and the materialize path removed from the POST 405 list
+   because the route now exists.
+4. Updated `tests/unit/viewerAnnotationContractPromotion.test.ts`: the earlier
+   "materialize-reference is 405" assertion now expects 400, because the route
+   exists and its closed body rejects `{}`.
+5. Updated `tests/unit/viewerPwaBuild.test.ts` (+1 test): the built service
+   worker still has one route with the `/api/` denylist, and precaches no
+   annotation or materialization path.
+
+## 20. Browser tests
+
+New `tests/browser/referenceAnnotationMaterialization.test.ts` has 4
+real-Chromium tests. It reuses `writeReferenceCandidateFixture`,
+`TestResources`, and the Prompt 4 interaction patterns. No new fixture
+framework was added.
+
+1. Create, through the real UI, in a project with `acceptance.reference`
+   configured:
+   1. draw a rectangle, create a candidate region, and confirm;
+   2. the unsaved message, disabled button, and warning are shown;
+   3. save, select, and materialize.
+   It then checks:
+   1. 201 with imported lifecycle, approval required, and supersession;
+   2. every success UI line;
+   3. the selection reset;
+   4. canonical regions;
+   5. exactly one new directory and no approved sibling;
+   6. byte-identical sources and project config;
+   7. the new handle in `/api/index` as `external-reference-imported`, while
+      the inspector still shows the original reference;
+   8. after reload, two imported references in the evidence list.
+2. Refine from an approved source: select `sidebar`, draw, refine, confirm,
+   save, and materialize. The new geometry is correct and `header` is
+   unchanged. Supersession is the approved id. Image bytes equal the owning
+   imported image. Both source directories are unchanged and there is still
+   one approved record.
+3. Create plus requirement, selected only: two confirmed creates and a
+   confirmed protected `region-a width` requirement with an
+   absolute-reference-px 4 tolerance. Selecting `region-a` and the
+   requirement materializes both. `region-b` is absent.
+4. Informational and candidate: confirmed inspect and asset-sensitive items
+   show the informational reason with disabled checkboxes. A candidate create
+   is disabled with its reason. Forced API requests for each return 422, and
+   no reference is written.
+5. `tests/browser/pwaHardening.test.ts` now also sends a POST to the
+   materialization route from the page. It returns 403 with `no-store`, and
+   cache storage holds zero `/api/` entries.
+6. Focused run of the new browser test plus the affected
+   `referenceAnnotationAuthoring`, `runtimeAnnotationContractPromotion`,
+   `referenceCandidateWorkspace`, `referenceBindingFidelityWorkspace`, and
+   `pwaHardening` suites: PASS (6 files, 53 tests).
+
+## 21. Regression validation
+
+Every command below was run on Windows in the repository root.
+
+1. Focused unit run of the two new unit suites plus
+   `externalReferencePersistenceService`, `externalReferenceRequirements`,
+   `externalReferenceRegions`, `externalReferenceRegionRelationships`,
+   `viewerReferenceAnnotationModel`, and `evidenceDiscoveryTempDirectories`:
+   PASS (8 files, 138 tests)
+2. `npm run typecheck`: PASS
+3. `npm run lint`: PASS. The first run reported one unused test helper, which
+   was removed. The rerun passed.
+4. `npm test`: PASS (84 files, 1378 tests)
+5. `npm run build`: PASS
+6. `npm run test:browser`: PASS (25 files, 231 tests, real Chromium)
+7. `npm run test:security`: PASS (unit: 16 files, 167 tests. Browser: 3 files,
+   77 tests)
+8. `npm run check:docs`: PASS (17 required files)
+9. `npm pack --dry-run`: PASS (359 files, 934.5 kB, version 0.8.1)
+
+## 22. Security regression
+
+1. All three authoring POST routes share one gate and one session write
+   queue. The Prompt 2 Host, Origin, token, body, content-type,
+   content-encoding, and method tests pass. The new route's own gate tests
+   pass.
+2. The browser supplies only item ids. It cannot choose the output location,
+   source root, image path, or project path. Unknown body fields return 400.
+3. Image bytes are read only through the existing `resolveMedia` containment
+   and symlink checks. The source root is resolved only through
+   `resolveContainedDir`.
+4. Responses contain no filesystem paths or token. Failures return bounded
+   messages and no stack traces.
+5. The PWA service worker does not cache the new route. This is proven both
+   at build level and in live Chromium cache storage.
+6. `package.json` (script only): `test:security` now also runs
+   `tests/unit/viewerAnnotationReferenceMaterialization.test.ts`. No version
+   or dependency changed.
+
+## 23. Temporary-directory hardening regression
+
+1. The canonical reference writer still uses its existing temporary directory
+   and atomic rename. Writer retry behavior was not changed.
+2. The Prompt 5 discovery rule that skips `.tmp-*` directories covers the
+   reference writer too. `tests/unit/evidenceDiscoveryTempDirectories.test.ts`
+   passes (2 tests), in the focused run and in `npm test` and
+   `npm run test:security`.
+3. Browser tests that materialize through the live viewer and then read
+   `/api/index` saw only complete artifacts.
+
+## 24. Scope audit
+
+1. external-reference schema changed: false
+2. annotation schema changed: false
+3. reference approval changed: false
+4. automatic reference approval added: false
+5. project reference acceptance updated: false
+6. project config schema changed: false
+7. alias catalog schema changed: false
+8. runtime contract promotion semantics changed: false
+9. ContractPrimitive vocabulary changed: false
+10. annotation evaluator added: false
+11. reference evaluator added: false
+12. package version changed: false
+13. dependency changed: false
+
+No file under `src/domain/` or `src/artifacts/` changed.
+`externalReferencePersistenceService.ts`,
+`visualAnnotationContractPromotionService.ts`, and `projectConfig.ts` are
+unchanged. The promotion route parser now calls the shared `itemIds` helper
+with identical rules and messages. Its tests pass unchanged except for the
+materialize assertion described in section 19. No broad documentation file
+was changed. `dist/`, `node_modules/`, and workflow state are not tracked.
+
+## 25. Changed files
+
+### 25.1 Added production files
+
+1. `src/application/visualAnnotationReferenceMaterializationService.ts`
+2. `src/viewerServer/annotationReferenceMaterialization.ts` (route use case,
+   keeps the HTTP layer thin)
+3. `viewer/src/hooks/useAnnotationReferenceMaterialization.ts`
+
+### 25.2 Modified production files
+
+1. `src/viewerServer/httpServer.ts` (route, status map, protocol 1.3.0)
+2. `src/viewerServer/annotationAuthoring.ts` (shared `parseAuthoringItemIds`
+   and `MAX_AUTHORING_ITEM_IDS`, so the new route reuses the Prompt 5 body
+   rule instead of copying it)
+3. `src/viewerServer/annotationContractPromotion.ts` (uses the shared helper,
+   behavior unchanged)
+4. `src/projectWorkflow/projectPaths.ts`
+5. `src/index.ts`
+6. `viewer/src/components/ReferenceAnnotationPanel.tsx`
+7. `viewer/src/components/ReferenceWorkspace.tsx` (wires the hook and the
+   in-memory token into the panel)
+8. `viewer/src/annotation/referenceIntent.ts` (presentation eligibility
+   helpers)
+9. `viewer/src/styles/index.css`
+10. `package.json` (`test:security` script only)
+
+### 25.3 Tests and report
+
+1. Added: `tests/unit/visualAnnotationReferenceMaterialization.test.ts`
+2. Added: `tests/unit/viewerAnnotationReferenceMaterialization.test.ts`
+3. Added: `tests/browser/referenceAnnotationMaterialization.test.ts`
+4. Modified: `tests/unit/viewerAuthoringSecurity.test.ts`
+5. Modified: `tests/unit/viewerAnnotationContractPromotion.test.ts`
+6. Modified: `tests/unit/viewerPwaBuild.test.ts`
+7. Modified: `tests/browser/pwaHardening.test.ts`
+8. Added: `docs/reports/v0.9-batch6-reference-materialization.md`
+
+### 25.4 Generated paths
+
+Test temporary directories were created under the OS temp directory by the
+existing test resource helpers and removed after each test. Validation logs
+were written to the session scratchpad, outside the repository. `npm run
+build` refreshed the ignored `dist/`. No workflow state was created.
+
+## 26. Remaining next step
+
+v0.9 Prompt 7: integrated acceptance, packaging, documentation, and
+regression. Prompt 7 owns broad documentation reconciliation for the new
+route, the protocol version 1.3.0, the project reference output location, and
+the materialization UI.