my-frontend-observer 0.4.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +60 -0
- package/README.md +39 -11
- package/dist/application/frontendContractEvaluationService.d.ts +49 -0
- package/dist/application/frontendContractEvaluationService.js +112 -0
- package/dist/application/frontendContractEvaluationService.js.map +1 -0
- package/dist/application/frontendContractPersistenceService.d.ts +56 -0
- package/dist/application/frontendContractPersistenceService.js +91 -0
- package/dist/application/frontendContractPersistenceService.js.map +1 -0
- package/dist/artifacts/comparisonArtifactReader.d.ts +18 -0
- package/dist/artifacts/comparisonArtifactReader.js +35 -0
- package/dist/artifacts/comparisonArtifactReader.js.map +1 -0
- package/dist/artifacts/frontendContractArtifactReader.d.ts +24 -0
- package/dist/artifacts/frontendContractArtifactReader.js +47 -0
- package/dist/artifacts/frontendContractArtifactReader.js.map +1 -0
- package/dist/artifacts/frontendContractArtifactWriter.d.ts +34 -0
- package/dist/artifacts/frontendContractArtifactWriter.js +70 -0
- package/dist/artifacts/frontendContractArtifactWriter.js.map +1 -0
- package/dist/artifacts/frontendContractEvaluationArtifactReader.d.ts +17 -0
- package/dist/artifacts/frontendContractEvaluationArtifactReader.js +34 -0
- package/dist/artifacts/frontendContractEvaluationArtifactReader.js.map +1 -0
- package/dist/artifacts/frontendContractEvaluationArtifactWriter.d.ts +32 -0
- package/dist/artifacts/frontendContractEvaluationArtifactWriter.js +58 -0
- package/dist/artifacts/frontendContractEvaluationArtifactWriter.js.map +1 -0
- package/dist/cli.js +496 -6
- package/dist/cli.js.map +1 -1
- package/dist/domain/frontendContractEvaluation.d.ts +57 -0
- package/dist/domain/frontendContractEvaluation.js +454 -0
- package/dist/domain/frontendContractEvaluation.js.map +1 -0
- package/dist/domain/frontendContractEvaluationArtifact.d.ts +65 -0
- package/dist/domain/frontendContractEvaluationArtifact.js +108 -0
- package/dist/domain/frontendContractEvaluationArtifact.js.map +1 -0
- package/dist/domain/frontendContractIdentity.d.ts +39 -0
- package/dist/domain/frontendContractIdentity.js +70 -0
- package/dist/domain/frontendContractIdentity.js.map +1 -0
- package/dist/domain/frontendContracts.d.ts +188 -0
- package/dist/domain/frontendContracts.js +260 -0
- package/dist/domain/frontendContracts.js.map +1 -0
- package/dist/index.d.ts +22 -0
- package/dist/index.js +12 -0
- package/dist/index.js.map +1 -1
- package/docs/ARCHITECTURE.md +98 -1
- package/docs/CI_CD.md +47 -0
- package/docs/COMMANDS.md +162 -0
- package/docs/CONTRACTS.md +175 -3
- package/docs/CURRENT_STATE.md +92 -10
- package/docs/DEVELOPMENT.md +58 -10
- package/docs/PROJECT_OVERVIEW.md +19 -13
- package/docs/QUICKSTART.md +5 -0
- package/docs/RELEASE.md +17 -10
- package/docs/ROADMAP.md +6 -0
- package/docs/SECURITY.md +13 -1
- package/docs/WORKFLOWS.md +65 -8
- package/package.json +1 -1
package/docs/COMMANDS.md
CHANGED
|
@@ -362,6 +362,168 @@ node dist/cli.js compare `
|
|
|
362
362
|
--config-file .\comparison-config.json
|
|
363
363
|
```
|
|
364
364
|
|
|
365
|
+
## `approve-baseline`
|
|
366
|
+
|
|
367
|
+
**Current status: shipped as part of the published `my-frontend-observer@0.5.0`
|
|
368
|
+
package.** Frontend contract schema is `1.0.0`, independent of the
|
|
369
|
+
observation (`1.2.0`) and comparison (`1.0.0`) schemas.
|
|
370
|
+
|
|
371
|
+
`my-frontend-observer approve-baseline` is the *only* baseline-approval
|
|
372
|
+
operation in the observer — approval is never inferred from a successful
|
|
373
|
+
comparison or evaluation, and no command automatically supersedes or selects
|
|
374
|
+
a baseline. It explicitly approves and persists one already-authored
|
|
375
|
+
`PersistentBaselineContract` against the observation it claims to approve:
|
|
376
|
+
|
|
377
|
+
```text
|
|
378
|
+
my-frontend-observer approve-baseline --observation <observation-artifact-root> --contract-file <json-file> --output <directory>
|
|
379
|
+
```
|
|
380
|
+
|
|
381
|
+
Required:
|
|
382
|
+
|
|
383
|
+
- `--observation <path>` — root directory of the persisted observation
|
|
384
|
+
artifact this baseline claims to approve (read through the existing
|
|
385
|
+
observation-artifact reader).
|
|
386
|
+
- `--contract-file <json-file>` — local JSON file containing one raw
|
|
387
|
+
`PersistentBaselineContract` (no wrapper field). As with `--config-file`,
|
|
388
|
+
only file readability/JSON-validity/non-array-object-root is checked here;
|
|
389
|
+
every structural rule (artifact kind, schema version, clause shape) is
|
|
390
|
+
enforced by the existing frozen domain validator.
|
|
391
|
+
- `--output <directory>` — portable, relative output location for the
|
|
392
|
+
baseline artifact.
|
|
393
|
+
|
|
394
|
+
Before persisting, the application layer verifies the contract's frozen
|
|
395
|
+
`sourceObservation` reference (`observationId`, `requestId`, `producer`,
|
|
396
|
+
`observationSchemaVersion`) actually matches the supplied observation
|
|
397
|
+
artifact's stable identity — approving a baseline against an unrelated
|
|
398
|
+
observation is rejected, even if both artifacts are individually valid. Any
|
|
399
|
+
`supersedesBaselineId` already authored in the contract is preserved exactly
|
|
400
|
+
as supplied; this command never discovers a prior baseline, infers
|
|
401
|
+
supersession, or deletes anything.
|
|
402
|
+
|
|
403
|
+
On success the command prints exactly:
|
|
404
|
+
|
|
405
|
+
```text
|
|
406
|
+
Baseline: <baseline-id>
|
|
407
|
+
State: approved
|
|
408
|
+
Artifact: <baseline-artifact-root>
|
|
409
|
+
Clauses: <count>
|
|
410
|
+
Supersedes: <baseline-id|none>
|
|
411
|
+
```
|
|
412
|
+
|
|
413
|
+
and exits `0`. It exits nonzero for invalid CLI syntax, an unreadable/
|
|
414
|
+
malformed/structurally-invalid contract file, a `PerChangeContract` passed
|
|
415
|
+
where a baseline is expected, a source-observation mismatch, an existing
|
|
416
|
+
artifact collision (baseline identities are never overwritten), or a failed
|
|
417
|
+
artifact write.
|
|
418
|
+
|
|
419
|
+
## `save-change-contract`
|
|
420
|
+
|
|
421
|
+
**Current status: shipped as part of the published `my-frontend-observer@0.5.0`
|
|
422
|
+
package.**
|
|
423
|
+
|
|
424
|
+
`my-frontend-observer save-change-contract` validates and persists one
|
|
425
|
+
already-authored `PerChangeContract` so it can later be evaluated — this is
|
|
426
|
+
persistence only, never approval:
|
|
427
|
+
|
|
428
|
+
```text
|
|
429
|
+
my-frontend-observer save-change-contract --contract-file <json-file> --output <directory>
|
|
430
|
+
```
|
|
431
|
+
|
|
432
|
+
Required:
|
|
433
|
+
|
|
434
|
+
- `--contract-file <json-file>` — local JSON file containing one raw
|
|
435
|
+
`PerChangeContract` (no wrapper field).
|
|
436
|
+
- `--output <directory>` — portable, relative output location for the
|
|
437
|
+
change-contract artifact.
|
|
438
|
+
|
|
439
|
+
Domain/application validation rejects a `PersistentBaselineContract` passed
|
|
440
|
+
here, malformed clauses, an unsupported authored category, an authored
|
|
441
|
+
`category: "unexpected"` (the derived-only fifth classification can never be
|
|
442
|
+
authored as a permission), and invalid tolerance/mode fields — none of this
|
|
443
|
+
is duplicated in CLI code. Any `supersedesBaselineClauseIds` already
|
|
444
|
+
authored on a clause is preserved exactly; resolving those references
|
|
445
|
+
against a particular baseline remains `evaluate-contract`'s responsibility.
|
|
446
|
+
|
|
447
|
+
On success the command prints exactly:
|
|
448
|
+
|
|
449
|
+
```text
|
|
450
|
+
Change contract: <contract-id>
|
|
451
|
+
Artifact: <contract-artifact-root>
|
|
452
|
+
Clauses: <count>
|
|
453
|
+
Supersedes baseline clauses: <count>
|
|
454
|
+
```
|
|
455
|
+
|
|
456
|
+
and exits `0`. It exits nonzero for invalid CLI syntax, an unreadable/
|
|
457
|
+
malformed/structurally-invalid contract file, a persistent baseline contract
|
|
458
|
+
passed here, an existing artifact collision, or a failed artifact write.
|
|
459
|
+
|
|
460
|
+
## `evaluate-contract`
|
|
461
|
+
|
|
462
|
+
**Current status: shipped as part of the published `my-frontend-observer@0.5.0`
|
|
463
|
+
package.** Frontend contract evaluation artifact schema is `1.0.0`, its own
|
|
464
|
+
independent family.
|
|
465
|
+
|
|
466
|
+
`my-frontend-observer evaluate-contract` executes the canonical Batch 2
|
|
467
|
+
evaluator against already-persisted evidence/contracts and persists the
|
|
468
|
+
result:
|
|
469
|
+
|
|
470
|
+
```text
|
|
471
|
+
my-frontend-observer evaluate-contract --before <observation-artifact-root> --after <observation-artifact-root> --comparison <comparison-artifact-root> --baseline <baseline-contract-artifact-root> --change <per-change-contract-artifact-root> --output <directory> [--enforce]
|
|
472
|
+
```
|
|
473
|
+
|
|
474
|
+
Required:
|
|
475
|
+
|
|
476
|
+
- `--before <path>` / `--after <path>` — root directories of the persisted
|
|
477
|
+
before/after observation artifacts.
|
|
478
|
+
- `--comparison <path>` — root directory of the already-persisted comparison
|
|
479
|
+
artifact for that before/after pair.
|
|
480
|
+
- `--baseline <path>` — root directory of the already-approved baseline
|
|
481
|
+
contract artifact.
|
|
482
|
+
- `--change <path>` — root directory of the already-persisted per-change
|
|
483
|
+
contract artifact.
|
|
484
|
+
- `--output <directory>` — portable, relative output location for the
|
|
485
|
+
evaluation artifact.
|
|
486
|
+
|
|
487
|
+
Options:
|
|
488
|
+
|
|
489
|
+
- `--enforce` — makes a `FAIL` verdict produce a nonzero process exit
|
|
490
|
+
status. A `FAIL` evaluation is always persisted and printed identically
|
|
491
|
+
with or without this flag; `--enforce` changes only the process exit code
|
|
492
|
+
— never evaluation identity, contents, or persistence.
|
|
493
|
+
|
|
494
|
+
**`evaluate-contract` never launches a browser, never re-resolves targets,
|
|
495
|
+
and never recomputes comparison or relationship evidence** — it reads the
|
|
496
|
+
already-persisted before/after observations and comparison exactly as given
|
|
497
|
+
(through the existing observation-artifact reader and a new comparison
|
|
498
|
+
reader) and calls the canonical `evaluateFrontendContract` exactly once.
|
|
499
|
+
|
|
500
|
+
A `FAIL` verdict (a found regression or unsatisfied contract clause) is a
|
|
501
|
+
successful, persisted evaluation outcome — not an execution error. On
|
|
502
|
+
success (evaluation constructed and persisted, verdict `PASS`, or verdict
|
|
503
|
+
`FAIL` without `--enforce`), the command prints exactly:
|
|
504
|
+
|
|
505
|
+
```text
|
|
506
|
+
Evaluation: <evaluation-id>
|
|
507
|
+
Verdict: <PASS|FAIL>
|
|
508
|
+
Artifact: <evaluation-artifact-root>
|
|
509
|
+
Clauses: <total-clause-result-count>
|
|
510
|
+
Unexpected: <unexpected-change-count>
|
|
511
|
+
Enforced: <yes|no>
|
|
512
|
+
```
|
|
513
|
+
|
|
514
|
+
and exits `0`; with `--enforce` and verdict `FAIL`, it prints the same
|
|
515
|
+
result and exits nonzero. It exits nonzero and persists nothing for invalid
|
|
516
|
+
CLI syntax or an unreadable/malformed/incoherent source artifact (evaluation
|
|
517
|
+
could not even be constructed) — distinct from a legitimate persisted `FAIL`.
|
|
518
|
+
|
|
519
|
+
The evaluation artifact directory contains `manifest.json` only — no
|
|
520
|
+
screenshot is copied. Operational `--before`/`--after`/`--comparison`/
|
|
521
|
+
`--baseline`/`--change`/`--output` paths never enter the persisted
|
|
522
|
+
`evaluationRequestId` or any other semantic field; two semantically
|
|
523
|
+
identical evaluations invoked from different filesystem locations share the
|
|
524
|
+
same `evaluationRequestId` even though each execution gets a fresh
|
|
525
|
+
`evaluationId`.
|
|
526
|
+
|
|
365
527
|
## Foundation commands
|
|
366
528
|
|
|
367
529
|
- `npm install` — install dependencies (includes the `playwright` runtime
|
package/docs/CONTRACTS.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
## Current contracts
|
|
4
4
|
|
|
5
|
-
The observation artifact contract is published as `my-frontend-observer@0.
|
|
5
|
+
The observation artifact contract is published as `my-frontend-observer@0.5.0`
|
|
6
6
|
and proven both from the source checkout and from the packed npm tarball,
|
|
7
7
|
on Windows, Linux, and macOS. The observation schema is `1.2.0` (see "v0.2
|
|
8
8
|
target contract" and "v0.3 scroll scenario contract" below):
|
|
@@ -250,6 +250,175 @@ The public entry point is `my-frontend-observer compare --before <root>
|
|
|
250
250
|
--after <root> --output <directory> [--config-file <json-file>]` (see
|
|
251
251
|
`docs/COMMANDS.md`) - comparison itself never launches a browser.
|
|
252
252
|
|
|
253
|
+
## v0.5 frontend contract and evaluation (shipped as part of this release)
|
|
254
|
+
|
|
255
|
+
Downstream of the v0.4 observation/comparison/relationship evidence above,
|
|
256
|
+
`src/domain/frontendContracts.ts` freezes the v0.5 contract/change-scope
|
|
257
|
+
model, `src/domain/frontendContractIdentity.ts` freezes deterministic
|
|
258
|
+
contract/baseline/clause identity, and `src/domain/frontendContractEvaluation.ts`
|
|
259
|
+
implements the one canonical pure evaluation engine. Baseline/per-change
|
|
260
|
+
contract persistence, evaluation-artifact persistence, explicit baseline
|
|
261
|
+
approval, and public CLI exposure are all implemented and shipped (see
|
|
262
|
+
"v0.5 contract and evaluation persistence" and "v0.5 public contract/
|
|
263
|
+
evaluation commands" below).
|
|
264
|
+
|
|
265
|
+
**Contract classes**: a `PersistentBaselineContract` (append/supersession-based
|
|
266
|
+
history via an optional `supersedesBaselineId`) and a `PerChangeContract`
|
|
267
|
+
(the allowed scope of one requested change). Both share `artifactKind:
|
|
268
|
+
"my-frontend-observer/frontend-contract"` and `schemaVersion: "1.0.0"` - an
|
|
269
|
+
independent family from the observation (`1.2.0`) and comparison (`1.0.0`)
|
|
270
|
+
schemas; the frontend-contract schema constant happens to share the version
|
|
271
|
+
string `1.0.0` with comparison's by coincidence only.
|
|
272
|
+
|
|
273
|
+
**Four authored categories, one derived classification**: every per-change
|
|
274
|
+
clause is authored as exactly one of `requested`, `expected-dependent`,
|
|
275
|
+
`protected`, or `preserved`. `unexpected` is a fifth, *derived-only*
|
|
276
|
+
classification the evaluator produces for a meaningful rendered difference no
|
|
277
|
+
active clause accounts for - it can never be authored as a permission.
|
|
278
|
+
|
|
279
|
+
**Bounded contract primitives**: 15 frozen `ContractPrimitive` kinds cover
|
|
280
|
+
visibility, clipping, width bounds, non-overlap, relative width, vertical
|
|
281
|
+
sequence, geometric fit (explicitly distinct from DOM containment),
|
|
282
|
+
document-width-vs-viewport, scroll ownership, initial-viewport position,
|
|
283
|
+
relationship-unchanged, and property-unchanged/increases/decreases - a closed
|
|
284
|
+
vocabulary, never a generic expression language.
|
|
285
|
+
|
|
286
|
+
**Contract tolerance**: `exact` / `absolute-px` / `percent`, independent of
|
|
287
|
+
`ComparisonConfig.geometryTolerancePx` (which only suppresses insignificant
|
|
288
|
+
comparison noise and is never contract authorization). Percent tolerance's
|
|
289
|
+
denominator is the absolute before-value.
|
|
290
|
+
|
|
291
|
+
**Required vs. permitted expected-dependent**: `required` clauses must occur
|
|
292
|
+
compliantly to pass; `permitted` clauses accept no change or a compliant
|
|
293
|
+
change, and fail only on a strictly contradictory change.
|
|
294
|
+
|
|
295
|
+
**Evaluation result vocabulary**: each clause resolves to `pass` / `fail` /
|
|
296
|
+
`unavailable` (with a required non-empty reason - required evidence gaps and
|
|
297
|
+
an `incomparable` source comparison never fabricate a `pass`) / `conflict`
|
|
298
|
+
(with at least two `conflictingClauseIds` - covers both an unresolved
|
|
299
|
+
baseline/per-change contradiction and an unknown `supersedesBaselineClauseIds`
|
|
300
|
+
reference). The overall verdict is `PASS` only when every clause result is
|
|
301
|
+
`pass` and no unexpected change remains; otherwise `FAIL` - there is no
|
|
302
|
+
partial-pass scoring.
|
|
303
|
+
|
|
304
|
+
**Explicit supersession, never inferred**: a per-change clause may list
|
|
305
|
+
`supersedesBaselineClauseIds` to remove specific baseline clauses from active
|
|
306
|
+
evaluation. Two clauses that structurally contradict each other on the same
|
|
307
|
+
(target, property) without explicit supersession produce a `conflict`, never
|
|
308
|
+
a silent preference for one side.
|
|
309
|
+
|
|
310
|
+
**Reuses existing v0.4 evidence directly**: the evaluator consumes an
|
|
311
|
+
already-computed `ComparisonArtifact` (`differences`, `relationshipChanges`,
|
|
312
|
+
`relationshipsBefore`/`relationshipsAfter`, `comparability`) and the source
|
|
313
|
+
`ObservationArtifact` pair - it never re-launches a browser, re-resolves a
|
|
314
|
+
target, or reimplements clipping/relationship/scroll-owner derivation.
|
|
315
|
+
Unexpected-change derivation reads `ComparisonArtifact.differences` only
|
|
316
|
+
(which already includes one difference per relationship change), so a single
|
|
317
|
+
logical transition is never double-counted.
|
|
318
|
+
|
|
319
|
+
## v0.5 contract and evaluation persistence (shipped as part of this release)
|
|
320
|
+
|
|
321
|
+
Persistence consumes the frozen v0.5 domain above; it never redefines it.
|
|
322
|
+
`src/artifacts/frontendContractArtifactWriter.ts`/`frontendContractArtifactReader.ts`
|
|
323
|
+
persist and read both `PersistentBaselineContract` and `PerChangeContract`
|
|
324
|
+
symmetrically (both already share `CONTRACT_ARTIFACT_KIND`/`CONTRACT_SCHEMA_VERSION`,
|
|
325
|
+
so one writer/reader pair serves both contract classes) as
|
|
326
|
+
`<outputLocation>/<baselineId|contractId>/manifest.json`, following the same
|
|
327
|
+
atomic-write discipline as `artifacts/artifactWriter.ts`/`artifacts/comparisonArtifactWriter.ts`
|
|
328
|
+
(sibling temporary directory, then one atomic rename; an existing directory at
|
|
329
|
+
the final identity is a genuine collision and is rejected, never overwritten -
|
|
330
|
+
prior baseline history is never rewritten). `src/artifacts/comparisonArtifactReader.ts`
|
|
331
|
+
is a new Batch 3 addition (no comparison reader existed before) mirroring
|
|
332
|
+
`artifacts/artifactReader.ts`'s discipline exactly, changing no comparison
|
|
333
|
+
semantics and keeping comparison schema `1.0.0`.
|
|
334
|
+
|
|
335
|
+
**Evaluation artifact envelope**: Batch 1 froze the evaluation-result
|
|
336
|
+
vocabulary (`ClauseEvaluationResult`, `OverallVerdict`) but not a persistable
|
|
337
|
+
envelope, so `src/domain/frontendContractEvaluationArtifact.ts` adds exactly
|
|
338
|
+
that - `artifactKind: "my-frontend-observer/frontend-contract-evaluation"`,
|
|
339
|
+
`schemaVersion: "1.0.0"` (its own independent family, distinct from
|
|
340
|
+
observation/comparison/frontend-contract), an `evaluationId`/`evaluationRequestId`
|
|
341
|
+
pair, bounded `before`/`after` source-observation references, and
|
|
342
|
+
`comparisonId`/`comparisonRequestId` plus `contracts: {baselineId,
|
|
343
|
+
contractId}` references - never an embedded `ObservationArtifact` or copied
|
|
344
|
+
screenshot. It reuses `ClauseEvaluationResult`/`OverallVerdict`/
|
|
345
|
+
`UnexpectedChangeResult` unchanged and contains no evaluation logic itself.
|
|
346
|
+
`evaluationRequestId` is a deterministic function of `{baselineId,
|
|
347
|
+
contractId, beforeObservationId, afterObservationId, comparisonRequestId}`
|
|
348
|
+
(`frontendContractIdentity.ts#buildFrontendContractEvaluationRequestIdentity` -
|
|
349
|
+
deliberately `comparisonRequestId`, not the fresh-per-execution
|
|
350
|
+
`comparisonId`, so semantically identical evaluations share an identity);
|
|
351
|
+
`evaluationId` reuses the existing generic `buildFrontendContractInstanceIdentity`
|
|
352
|
+
unchanged. `src/artifacts/frontendContractEvaluationArtifactWriter.ts`/
|
|
353
|
+
`frontendContractEvaluationArtifactReader.ts` persist/read it with the same
|
|
354
|
+
atomic-write discipline as above.
|
|
355
|
+
|
|
356
|
+
**Application seam**: `src/application/frontendContractEvaluationService.ts#evaluateAndPersist`
|
|
357
|
+
calls the existing pure `evaluateFrontendContract` exactly once and - only
|
|
358
|
+
for a structurally constructible result, whether the verdict is `PASS` or
|
|
359
|
+
`FAIL` - persists exactly one evaluation artifact; an `{ok: false}` evaluator
|
|
360
|
+
result (evidence could not be constructed into an evaluation at all) is never
|
|
361
|
+
persisted as a fabricated artifact. `evaluateAndPersistFromArtifactRoots` is
|
|
362
|
+
the future-CLI-facing wrapper: it reads two observations through the existing
|
|
363
|
+
`readObservationArtifact` (never a second observation reader), the
|
|
364
|
+
comparison and the two contracts through the readers above, then delegates
|
|
365
|
+
to `evaluateAndPersist` exactly once.
|
|
366
|
+
|
|
367
|
+
## v0.5 public contract/evaluation commands (shipped as part of this release)
|
|
368
|
+
|
|
369
|
+
Three public commands expose the persistence/evaluation contract above (see
|
|
370
|
+
`docs/COMMANDS.md` for exact flags/output/exit behavior, not duplicated
|
|
371
|
+
here):
|
|
372
|
+
|
|
373
|
+
- `approve-baseline` → `frontendContractPersistenceService.ts#approveAndPersistBaseline`
|
|
374
|
+
→ validates a `PersistentBaselineContract` and its `sourceObservation`
|
|
375
|
+
coherence against a supplied observation artifact → persists via
|
|
376
|
+
`frontendContractArtifactWriter.ts`. The only baseline-approval act in the
|
|
377
|
+
observer.
|
|
378
|
+
- `save-change-contract` → `frontendContractPersistenceService.ts#persistPerChangeContract`
|
|
379
|
+
→ validates a `PerChangeContract` (rejecting a baseline contract, an
|
|
380
|
+
authored `unexpected` category, or any other structural violation) →
|
|
381
|
+
persists via the same writer. Persistence only, never approval.
|
|
382
|
+
- `evaluate-contract` → `frontendContractEvaluationService.ts#evaluateAndPersistFromArtifactRoots`
|
|
383
|
+
→ `evaluateFrontendContract` exactly once → `frontendContractEvaluationArtifactWriter.ts`
|
|
384
|
+
exactly once. `--enforce` affects only the process exit status for an
|
|
385
|
+
already-persisted `FAIL` verdict.
|
|
386
|
+
|
|
387
|
+
No command infers baseline approval or supersession automatically - not
|
|
388
|
+
`compare`, not a `PASS` evaluation, not any artifact writer.
|
|
389
|
+
|
|
390
|
+
This full command sequence is proven against real Chromium observations (not
|
|
391
|
+
hand-constructed artifacts) - see "v0.5 real-browser workflow proof" below.
|
|
392
|
+
|
|
393
|
+
## v0.5 real-browser workflow proof (shipped as part of this release)
|
|
394
|
+
|
|
395
|
+
`tests/browser/cliFrontendContracts.test.ts` and
|
|
396
|
+
`scripts/dev/builtCliFrontendContractsBrowserSmoke.mjs` drive the complete
|
|
397
|
+
`observe` → `approve-baseline` → `save-change-contract` → `observe` →
|
|
398
|
+
`compare` → `evaluate-contract` sequence against a real disposable local HTTP
|
|
399
|
+
fixture and real Chromium, proving two scenarios:
|
|
400
|
+
|
|
401
|
+
- a fully successful contract change - a real observed navigation-width
|
|
402
|
+
decrease and workspace-width increase, both satisfying their authored
|
|
403
|
+
`requested`/`expected-dependent` clauses, an unchanged `protected` rail
|
|
404
|
+
width, and an unclipped `preserved` navigation - overall `PASS`;
|
|
405
|
+
- the "milestone signature" failure - the same locally successful requested
|
|
406
|
+
change (navigation shrinks, workspace expands, both still `pass`)
|
|
407
|
+
co-occurring with a genuine `protected` right-rail width regression (a real
|
|
408
|
+
`resized` comparison difference) and a genuine `preserved` navigation
|
|
409
|
+
clipping regression (a real `clipping-changed` difference, `not-clipped` →
|
|
410
|
+
`clipped`) - overall `FAIL`.
|
|
411
|
+
|
|
412
|
+
Both scenarios confirm: `--enforce` changes only the process exit status
|
|
413
|
+
(`0` without it, nonzero with it) for the identical persisted
|
|
414
|
+
`evaluationRequestId`/`clauseResults`; every source observation and
|
|
415
|
+
comparison artifact is byte-identical before and after evaluation; the
|
|
416
|
+
evaluation directory contains `manifest.json` only (no copied screenshot);
|
|
417
|
+
and no operational filesystem path is ever serialized into a persisted
|
|
418
|
+
manifest. This is real-browser evidence layered on top of the CLI-level
|
|
419
|
+
proof in `tests/unit/cliFrontendContracts.test.ts` and the Chromium-free
|
|
420
|
+
`scripts/dev/builtCliFrontendContractsSmoke.mjs` - it does not replace them.
|
|
421
|
+
|
|
253
422
|
## Approved v0.1 design inputs
|
|
254
423
|
|
|
255
424
|
The historical greenfield scaffold plan recorded these v0.1 design decisions:
|
|
@@ -269,7 +438,10 @@ the file layout: there is no separate `evidence.json` - page/target evidence
|
|
|
269
438
|
is embedded directly inside `manifest.json`.
|
|
270
439
|
|
|
271
440
|
Comparison and relationship contracts belong to v0.4, and canonical
|
|
272
|
-
change-scope contracts belong to v0.5.
|
|
273
|
-
|
|
441
|
+
change-scope contracts belong to v0.5 - see "v0.5 frontend contract and
|
|
442
|
+
evaluation" above for the full shipped contract model, identity, evaluation
|
|
443
|
+
engine, persistence, baseline approval, and CLI exposure.
|
|
444
|
+
Bounded agent-context plus ecosystem integration contracts move to v0.6,
|
|
445
|
+
followed by the text/config-driven
|
|
274
446
|
coding-agent review contract in v0.7. Viewer and annotation contracts follow in
|
|
275
447
|
v0.8 and v0.9 and converge with the existing workflow in v0.10.
|
package/docs/CURRENT_STATE.md
CHANGED
|
@@ -1,8 +1,9 @@
|
|
|
1
1
|
# Current State
|
|
2
2
|
|
|
3
|
-
The project is published at package version `0.
|
|
4
|
-
|
|
5
|
-
|
|
3
|
+
The project is published at package version `0.5.0` (roadmap v0.5,
|
|
4
|
+
Executable Frontend Contracts and Explicit Change Scope; observation schema
|
|
5
|
+
`1.2.0`; comparison schema `1.0.0`; frontend contract schema `1.0.0`;
|
|
6
|
+
evaluation artifact schema `1.0.0`).
|
|
6
7
|
|
|
7
8
|
## Greenfield foundation established
|
|
8
9
|
|
|
@@ -314,15 +315,96 @@ evidence, comparison persistence, and public `compare` CLI are all
|
|
|
314
315
|
implemented, exercised end-to-end, packed-validated cross-platform, and
|
|
315
316
|
released.
|
|
316
317
|
|
|
318
|
+
v0.5 frontend contract model, identity, and evaluation engine are released
|
|
319
|
+
as part of `0.5.0`: `src/domain/frontendContracts.ts` (persistent baseline /
|
|
320
|
+
per-change contract types, the four authored change-scope categories plus
|
|
321
|
+
the derived `unexpected` classification, the 15-primitive bounded
|
|
322
|
+
vocabulary, contract tolerance, and the clause-result/overall-verdict
|
|
323
|
+
vocabulary), `src/domain/frontendContractIdentity.ts` (deterministic
|
|
324
|
+
contract/baseline/clause identity), and
|
|
325
|
+
`src/domain/frontendContractEvaluation.ts#evaluateFrontendContract`
|
|
326
|
+
(the one canonical pure evaluator: active-baseline/supersession calculation,
|
|
327
|
+
bounded conflict detection, per-category clause evaluation, difference-to-
|
|
328
|
+
scope matching, unexpected-change derivation, and overall PASS/FAIL) - all
|
|
329
|
+
covered by focused unit tests. Observation schema stays `1.2.0`, comparison
|
|
330
|
+
schema stays `1.0.0`.
|
|
331
|
+
|
|
332
|
+
v0.5 contract and evaluation persistence are also released as part of
|
|
333
|
+
`0.5.0`: `src/artifacts/frontendContractArtifactWriter.ts`/`frontendContractArtifactReader.ts`
|
|
334
|
+
(symmetric baseline/per-change contract persistence, atomic write, no
|
|
335
|
+
overwrite of existing history), `src/artifacts/comparisonArtifactReader.ts`
|
|
336
|
+
(new - no comparison reader existed before this batch; comparison schema
|
|
337
|
+
still `1.0.0`), `src/domain/frontendContractEvaluationArtifact.ts` (minimal
|
|
338
|
+
additive persisted envelope around the frozen evaluation-result vocabulary,
|
|
339
|
+
its own independent schema family `1.0.0`) with
|
|
340
|
+
`src/artifacts/frontendContractEvaluationArtifactWriter.ts`/`...Reader.ts`,
|
|
341
|
+
and `src/application/frontendContractEvaluationService.ts#evaluateAndPersist`/
|
|
342
|
+
`evaluateAndPersistFromArtifactRoots` (calls `evaluateFrontendContract`
|
|
343
|
+
exactly once, persists exactly one evaluation artifact for both `PASS` and
|
|
344
|
+
`FAIL` verdicts, never persists a fabricated artifact when evaluation
|
|
345
|
+
construction itself fails). `evaluateFrontendContract` itself is unmodified.
|
|
346
|
+
|
|
347
|
+
v0.5 public contract/baseline-approval/evaluation CLI is also released as
|
|
348
|
+
part of `0.5.0`:
|
|
349
|
+
`approve-baseline` (the only baseline-approval act - explicit only, never
|
|
350
|
+
inferred from `compare` or a `PASS` evaluation; verifies the contract's
|
|
351
|
+
`sourceObservation` matches the supplied observation before persisting),
|
|
352
|
+
`save-change-contract` (persistence only), and `evaluate-contract`
|
|
353
|
+
(evaluates already-persisted before/after/comparison/baseline/change
|
|
354
|
+
evidence exactly once and persists exactly one evaluation artifact;
|
|
355
|
+
`--enforce` makes a `FAIL` verdict exit nonzero without changing the
|
|
356
|
+
verdict, its identity, or its persisted content - a `FAIL` without
|
|
357
|
+
`--enforce` still exits `0`). `src/application/frontendContractPersistenceService.ts`
|
|
358
|
+
adds the two new thin application seams (`approveAndPersistBaseline`,
|
|
359
|
+
`persistPerChangeContract`); `src/cli.ts` gained no browser or artifact-
|
|
360
|
+
writer import. Covered by `tests/unit/cliFrontendContracts.test.ts` and the
|
|
361
|
+
Chromium-free `scripts/dev/builtCliFrontendContractsSmoke.mjs` dev smoke.
|
|
362
|
+
Observation schema `1.2.0`; comparison schema `1.0.0`; frontend contract
|
|
363
|
+
schema `1.0.0`; evaluation artifact schema `1.0.0` - no schema was bumped
|
|
364
|
+
to add this CLI.
|
|
365
|
+
|
|
366
|
+
v0.5 proved the complete public contract workflow (`observe` →
|
|
367
|
+
`approve-baseline` → `save-change-contract` → `observe` → `compare` →
|
|
368
|
+
`evaluate-contract`) against real Chromium observations, not hand-constructed
|
|
369
|
+
artifacts: a fully successful contract change (all clauses `pass`, overall
|
|
370
|
+
`PASS`), and the "milestone signature" case - a locally successful requested
|
|
371
|
+
change (navigation shrinks, workspace expands, both real and both `pass`)
|
|
372
|
+
coexisting with a genuine protected-property regression (real right-rail
|
|
373
|
+
`resized` difference) and a genuine preserved-invariant regression (real
|
|
374
|
+
`clipping-changed` difference, `not-clipped` → `clipped`) - producing overall
|
|
375
|
+
`FAIL`. Both scenarios are covered by `tests/browser/cliFrontendContracts.test.ts`
|
|
376
|
+
(real Chromium, via `tests/fixtures/server.ts`'s new `/contract` route) and by
|
|
377
|
+
the built-CLI dev smoke `scripts/dev/builtCliFrontendContractsBrowserSmoke.mjs`
|
|
378
|
+
(the built `dist/cli.js`, not the imported `runCli()`, against its own
|
|
379
|
+
disposable local HTTP fixture). Both confirm `--enforce` behavior (`FAIL`
|
|
380
|
+
persists and exits `0` without it, exits nonzero with it, identical
|
|
381
|
+
`evaluationRequestId` and `clauseResults` in both cases), full source
|
|
382
|
+
observation/comparison immutability, no screenshot copied into the
|
|
383
|
+
evaluation artifact, and no operational filesystem path leaked into any
|
|
384
|
+
persisted manifest.
|
|
385
|
+
|
|
386
|
+
The packed-readiness coverage gap this left (`V0_5_READINESS_VALIDATION_GAP_EXISTS`)
|
|
387
|
+
was corrected and proven cross-platform before release:
|
|
388
|
+
`scripts/ci/runPackedObservationSmoke.mjs` also exercises the installed
|
|
389
|
+
packed candidate's `approve-baseline`/`save-change-contract`/
|
|
390
|
+
`evaluate-contract` commands against real installed-candidate `observe`/
|
|
391
|
+
`compare` evidence, proving the same successful-change and milestone-
|
|
392
|
+
signature scenarios through the installed tarball rather than the source
|
|
393
|
+
checkout. v0.5 pre-release readiness passed on the validation branch
|
|
394
|
+
`validation/v0.5-pre-release` (GitHub Actions run `31727856546`, one shared
|
|
395
|
+
hash-verified candidate tarball on Windows, Linux, and macOS - see
|
|
396
|
+
`docs/CI_CD.md` for full evidence) before the version `0.5.0` release below.
|
|
397
|
+
|
|
317
398
|
## Not implemented
|
|
318
399
|
|
|
319
|
-
- v0.5
|
|
320
|
-
|
|
321
|
-
my-dev-kit runtime/static
|
|
322
|
-
integration, viewer, and annotation
|
|
400
|
+
- v0.5 baseline-selection/discovery policy (the caller must supply which
|
|
401
|
+
baseline to approve/evaluate against; there is no "find the current
|
|
402
|
+
baseline" command), source ownership, my-dev-kit runtime/static
|
|
403
|
+
integration, orchestrator/lab product integration, viewer, and annotation
|
|
404
|
+
all remain unimplemented.
|
|
323
405
|
|
|
324
406
|
## Next target
|
|
325
407
|
|
|
326
|
-
v0.1-v0.
|
|
327
|
-
`0.3.0`, `0.4.0`). v0.
|
|
328
|
-
|
|
408
|
+
v0.1-v0.5 are implemented, validated, and released (`0.1.0`, `0.2.0`,
|
|
409
|
+
`0.3.0`, `0.4.0`, `0.5.0`). v0.6 (Bounded Agent Context and Native
|
|
410
|
+
my-dev-kit Ecosystem Integration) is next.
|
package/docs/DEVELOPMENT.md
CHANGED
|
@@ -21,9 +21,9 @@ npm run check:docs
|
|
|
21
21
|
npm pack --dry-run
|
|
22
22
|
```
|
|
23
23
|
|
|
24
|
-
`npm test` runs the fast unit suite only (`tests/unit/`, currently
|
|
24
|
+
`npm test` runs the fast unit suite only (`tests/unit/`, currently 515
|
|
25
25
|
passing tests). `npm run test:browser` runs the real-Chromium integration
|
|
26
|
-
suite (`tests/browser/`, currently
|
|
26
|
+
suite (`tests/browser/`, currently 120 passing tests) against deterministic
|
|
27
27
|
local fixtures under `tests/fixtures/` and requires the Chromium binary
|
|
28
28
|
above to be installed first; it is kept out of `npm test` because it
|
|
29
29
|
launches a real browser and is slower.
|
|
@@ -114,13 +114,61 @@ before and after the comparison ran. Run it locally after `npm run build`:
|
|
|
114
114
|
node scripts/dev/builtCliCompareSmoke.mjs
|
|
115
115
|
```
|
|
116
116
|
|
|
117
|
-
|
|
117
|
+
`scripts/dev/builtCliFrontendContractsSmoke.mjs` is the v0.5 equivalent,
|
|
118
|
+
added alongside the `approve-baseline`/`save-change-contract`/
|
|
119
|
+
`evaluate-contract` command implementations (shipped as part of the
|
|
120
|
+
published `0.5.0` package - see `docs/CURRENT_STATE.md`). Unlike the other dev smokes, it needs no Chromium
|
|
121
|
+
at all: it hand-writes deterministic, schema-`1.2.0`-valid observation
|
|
122
|
+
manifests directly to a temporary directory (preserving the public
|
|
123
|
+
observation artifact contract without a real browser capture), then runs
|
|
124
|
+
the built `dist/cli.js` for `compare`, `approve-baseline`,
|
|
125
|
+
`save-change-contract`, and `evaluate-contract` - twice for the final
|
|
126
|
+
step, once without `--enforce` and once with it, against the same
|
|
127
|
+
milestone-signature contract (requested navigation shrink = pass, expected
|
|
128
|
+
workspace expansion = pass, protected right-rail width = fail, preserved
|
|
129
|
+
unclipped navigation = fail, overall = `FAIL`). It validates both exit
|
|
130
|
+
codes (`0` without `--enforce`, nonzero with it), that both invocations
|
|
131
|
+
persist byte-for-byte semantically identical evaluation evidence
|
|
132
|
+
(`evaluationRequestId` and `clauseResults` equal), that the evaluation
|
|
133
|
+
directory contains `manifest.json` only, that no operational filesystem
|
|
134
|
+
path leaked into either persisted evaluation manifest, and that every
|
|
135
|
+
source observation/comparison/contract artifact remains unmodified. Run it
|
|
136
|
+
locally after `npm run build`:
|
|
137
|
+
|
|
138
|
+
```powershell
|
|
139
|
+
node scripts/dev/builtCliFrontendContractsSmoke.mjs
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
`scripts/dev/builtCliFrontendContractsBrowserSmoke.mjs` is the v0.5 Batch 5
|
|
143
|
+
real-browser equivalent, added alongside
|
|
144
|
+
`tests/browser/cliFrontendContracts.test.ts`. Unlike the Chromium-free
|
|
145
|
+
`scripts/dev/builtCliFrontendContractsSmoke.mjs` above, this one launches a
|
|
146
|
+
real disposable local HTTP fixture and real Playwright Chromium, then drives
|
|
147
|
+
the built `dist/cli.js` through the complete `observe` → `approve-baseline`
|
|
148
|
+
→ `save-change-contract` → `observe` → `compare` → `evaluate-contract`
|
|
149
|
+
sequence twice: once against a candidate whose served content produces a
|
|
150
|
+
fully successful contract change (all clauses `pass`, overall `PASS`), and
|
|
151
|
+
once against a candidate that reproduces the milestone-signature failure - a
|
|
152
|
+
real observed navigation clipping regression and a real observed right-rail
|
|
153
|
+
width regression alongside an otherwise-successful requested/expected-
|
|
154
|
+
dependent change (overall `FAIL`). It validates the same `--enforce`
|
|
155
|
+
exit-code/identity behavior, screenshot-free evaluation directory, source
|
|
156
|
+
immutability, and path-privacy properties as the Chromium-free smoke, but
|
|
157
|
+
against genuine rendered geometry instead of hand-constructed artifacts. Run
|
|
158
|
+
it locally after `npm run build` (Chromium must already be installed):
|
|
159
|
+
|
|
160
|
+
```powershell
|
|
161
|
+
node scripts/dev/builtCliFrontendContractsBrowserSmoke.mjs
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
Unlike `scripts/ci/runPackedObservationSmoke.mjs`, none of these five dev
|
|
118
165
|
smokes is wired into any CI workflow or is a release gate - they are
|
|
119
166
|
source-checkout development evidence only, proving the built CLI's
|
|
120
|
-
`--targets-file`/`--scroll-scenario-file`/`compare
|
|
121
|
-
installing a packed tarball or requiring
|
|
122
|
-
None is part of the published package.
|
|
123
|
-
|
|
124
|
-
`scripts/ci/runPackedObservationSmoke.mjs`'s
|
|
125
|
-
`docs/CI_CD.md`) - the same script, against the same
|
|
126
|
-
tarball per platform.
|
|
167
|
+
`--targets-file`/`--scroll-scenario-file`/`compare`/frontend-contract
|
|
168
|
+
command behavior without installing a packed tarball or requiring
|
|
169
|
+
cross-platform infrastructure. None is part of the published package.
|
|
170
|
+
Cross-platform packed validation of the v0.1-v0.5 observation/compare/
|
|
171
|
+
contract behavior is `scripts/ci/runPackedObservationSmoke.mjs`'s
|
|
172
|
+
responsibility (see `docs/CI_CD.md`) - the same script, against the same
|
|
173
|
+
single candidate tarball per platform, now including the v0.5 contract/
|
|
174
|
+
evaluation CLI.
|
package/docs/PROJECT_OVERVIEW.md
CHANGED
|
@@ -17,19 +17,25 @@ The responsibility split is stable:
|
|
|
17
17
|
|
|
18
18
|
v0.1, Runtime Observation Foundation; v0.2, Stable Semantic Targets and
|
|
19
19
|
Region Identity; v0.3, Runtime Scrolling, Overflow, and Visibility Behavior;
|
|
20
|
-
|
|
21
|
-
Comparison
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
bounded
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
20
|
+
v0.4, Layout Relationships, Dependency Evidence, and Before/After
|
|
21
|
+
Comparison; and v0.5, Executable Frontend Contracts and Explicit Change
|
|
22
|
+
Scope, are released, published to npm (current version `0.5.0`, observation
|
|
23
|
+
schema `1.2.0`, comparison schema `1.0.0`, frontend contract schema `1.0.0`,
|
|
24
|
+
evaluation artifact schema `1.0.0`) and validated as a packed npm tarball in
|
|
25
|
+
a clean consumer environment across Windows, Linux, and macOS: a real
|
|
26
|
+
`observe` CLI command launches Chromium, enforces loopback-only safety,
|
|
27
|
+
captures bounded page/target evidence via legacy CSS-shorthand targets,
|
|
28
|
+
structured semantic `--targets-file` targets, or a bounded
|
|
29
|
+
`--scroll-scenario-file` runtime scroll scenario (`window-scroll-by` or
|
|
30
|
+
`target-scroll-by`), and persists one portable local artifact; a real
|
|
31
|
+
`compare` CLI command reads two already-persisted observation artifacts and
|
|
32
|
+
derives before/after layout-relationship and difference evidence without
|
|
33
|
+
launching a browser; and the public `approve-baseline`/`save-change-contract`/
|
|
34
|
+
`evaluate-contract` commands turn a persistent baseline contract plus a
|
|
35
|
+
per-change contract (requested/expected-dependent/protected/preserved scope,
|
|
36
|
+
plus the derived-only `unexpected` classification) into one canonical
|
|
37
|
+
`PASS`/`FAIL` evaluation, proven against real Chromium observations - see
|
|
38
|
+
`docs/CURRENT_STATE.md` for the implementation summary. v0.6–v0.10 remain
|
|
33
39
|
future and unimplemented.
|
|
34
40
|
|
|
35
41
|
The revised dependency path reaches practical coding-agent use before graphical
|
package/docs/QUICKSTART.md
CHANGED
|
@@ -30,6 +30,11 @@ Once you have two such artifacts, `node dist/cli.js compare --before
|
|
|
30
30
|
between them without launching a browser again - see
|
|
31
31
|
[COMMANDS.md](COMMANDS.md#compare) for details.
|
|
32
32
|
|
|
33
|
+
You can then approve a baseline, save a per-change contract, and evaluate a
|
|
34
|
+
candidate change against them plus the observation/comparison evidence
|
|
35
|
+
above - see [COMMANDS.md](COMMANDS.md#approve-baseline) for the exact flags
|
|
36
|
+
and [WORKFLOWS.md](WORKFLOWS.md) for the full flow.
|
|
37
|
+
|
|
33
38
|
To validate the repository itself instead:
|
|
34
39
|
|
|
35
40
|
```powershell
|