reflection-check 0.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +55 -0
- package/dist/adapters/route-manifest.d.ts +3 -0
- package/dist/adapters/route-manifest.js +98 -0
- package/dist/adapters/route-manifest.js.map +1 -0
- package/dist/cli.d.ts +3 -0
- package/dist/cli.js +93 -0
- package/dist/cli.js.map +1 -0
- package/dist/commands/doctor.d.ts +4 -0
- package/dist/commands/doctor.js +5 -0
- package/dist/commands/doctor.js.map +1 -0
- package/dist/commands/gc.d.ts +8 -0
- package/dist/commands/gc.js +45 -0
- package/dist/commands/gc.js.map +1 -0
- package/dist/commands/review.d.ts +7 -0
- package/dist/commands/review.js +149 -0
- package/dist/commands/review.js.map +1 -0
- package/dist/commands/run.d.ts +10 -0
- package/dist/commands/run.js +168 -0
- package/dist/commands/run.js.map +1 -0
- package/dist/commands/update.d.ts +11 -0
- package/dist/commands/update.js +183 -0
- package/dist/commands/update.js.map +1 -0
- package/dist/contracts/browser/assertions.d.ts +34 -0
- package/dist/contracts/browser/assertions.js +87 -0
- package/dist/contracts/browser/assertions.js.map +1 -0
- package/dist/contracts/browser/browser-contract.d.ts +13 -0
- package/dist/contracts/browser/browser-contract.js +35 -0
- package/dist/contracts/browser/browser-contract.js.map +1 -0
- package/dist/contracts/browser/console-observer.d.ts +6 -0
- package/dist/contracts/browser/console-observer.js +14 -0
- package/dist/contracts/browser/console-observer.js.map +1 -0
- package/dist/contracts/browser/overflow-check.d.ts +6 -0
- package/dist/contracts/browser/overflow-check.js +15 -0
- package/dist/contracts/browser/overflow-check.js.map +1 -0
- package/dist/contracts/browser/route-runner.d.ts +21 -0
- package/dist/contracts/browser/route-runner.js +98 -0
- package/dist/contracts/browser/route-runner.js.map +1 -0
- package/dist/contracts/component/component-visual-contract.d.ts +30 -0
- package/dist/contracts/component/component-visual-contract.js +147 -0
- package/dist/contracts/component/component-visual-contract.js.map +1 -0
- package/dist/contracts/design/command-adapter.d.ts +17 -0
- package/dist/contracts/design/command-adapter.js +60 -0
- package/dist/contracts/design/command-adapter.js.map +1 -0
- package/dist/contracts/design/design-contract.d.ts +8 -0
- package/dist/contracts/design/design-contract.js +149 -0
- package/dist/contracts/design/design-contract.js.map +1 -0
- package/dist/contracts/visual/baseline-compare.d.ts +19 -0
- package/dist/contracts/visual/baseline-compare.js +94 -0
- package/dist/contracts/visual/baseline-compare.js.map +1 -0
- package/dist/contracts/visual/image-diff.d.ts +27 -0
- package/dist/contracts/visual/image-diff.js +58 -0
- package/dist/contracts/visual/image-diff.js.map +1 -0
- package/dist/contracts/visual/thresholds.d.ts +15 -0
- package/dist/contracts/visual/thresholds.js +11 -0
- package/dist/contracts/visual/thresholds.js.map +1 -0
- package/dist/contracts/visual/visual-contract.d.ts +11 -0
- package/dist/contracts/visual/visual-contract.js +32 -0
- package/dist/contracts/visual/visual-contract.js.map +1 -0
- package/dist/core/artifact-store.d.ts +18 -0
- package/dist/core/artifact-store.js +105 -0
- package/dist/core/artifact-store.js.map +1 -0
- package/dist/core/baseline-store.d.ts +18 -0
- package/dist/core/baseline-store.js +56 -0
- package/dist/core/baseline-store.js.map +1 -0
- package/dist/core/config.d.ts +129 -0
- package/dist/core/config.js +159 -0
- package/dist/core/config.js.map +1 -0
- package/dist/core/define-reflection.d.ts +2 -0
- package/dist/core/define-reflection.js +4 -0
- package/dist/core/define-reflection.js.map +1 -0
- package/dist/core/exit-codes.d.ts +7 -0
- package/dist/core/exit-codes.js +9 -0
- package/dist/core/exit-codes.js.map +1 -0
- package/dist/core/failure-classifier.d.ts +3 -0
- package/dist/core/failure-classifier.js +19 -0
- package/dist/core/failure-classifier.js.map +1 -0
- package/dist/core/gc.d.ts +19 -0
- package/dist/core/gc.js +161 -0
- package/dist/core/gc.js.map +1 -0
- package/dist/core/manifest.d.ts +23 -0
- package/dist/core/manifest.js +21 -0
- package/dist/core/manifest.js.map +1 -0
- package/dist/core/redaction.d.ts +3 -0
- package/dist/core/redaction.js +63 -0
- package/dist/core/redaction.js.map +1 -0
- package/dist/core/report-schema.d.ts +262 -0
- package/dist/core/report-schema.js +112 -0
- package/dist/core/report-schema.js.map +1 -0
- package/dist/core/report-writer.d.ts +4 -0
- package/dist/core/report-writer.js +77 -0
- package/dist/core/report-writer.js.map +1 -0
- package/dist/core/server-manager.d.ts +23 -0
- package/dist/core/server-manager.js +64 -0
- package/dist/core/server-manager.js.map +1 -0
- package/dist/core/target-ir.d.ts +64 -0
- package/dist/core/target-ir.js +85 -0
- package/dist/core/target-ir.js.map +1 -0
- package/dist/index.d.ts +2 -0
- package/dist/index.js +2 -0
- package/dist/index.js.map +1 -0
- package/dist/integrations/playwright/browser-manager.d.ts +2 -0
- package/dist/integrations/playwright/browser-manager.js +5 -0
- package/dist/integrations/playwright/browser-manager.js.map +1 -0
- package/dist/integrations/playwright/context-factory.d.ts +7 -0
- package/dist/integrations/playwright/context-factory.js +19 -0
- package/dist/integrations/playwright/context-factory.js.map +1 -0
- package/dist/integrations/playwright/trace-policy.d.ts +5 -0
- package/dist/integrations/playwright/trace-policy.js +7 -0
- package/dist/integrations/playwright/trace-policy.js.map +1 -0
- package/dist/integrations/storybook/index-json.d.ts +21 -0
- package/dist/integrations/storybook/index-json.js +44 -0
- package/dist/integrations/storybook/index-json.js.map +1 -0
- package/dist/integrations/storybook/server.d.ts +8 -0
- package/dist/integrations/storybook/server.js +23 -0
- package/dist/integrations/storybook/server.js.map +1 -0
- package/dist/integrations/storybook/story-url.d.ts +2 -0
- package/dist/integrations/storybook/story-url.js +13 -0
- package/dist/integrations/storybook/story-url.js.map +1 -0
- package/dist/utils/process.d.ts +9 -0
- package/dist/utils/process.js +69 -0
- package/dist/utils/process.js.map +1 -0
- package/docs/agent-workflows.md +146 -0
- package/docs/artifacts-and-gc.md +125 -0
- package/docs/browser-contract.md +98 -0
- package/docs/ci.md +44 -0
- package/docs/configuration.md +210 -0
- package/docs/getting-started.md +166 -0
- package/docs/plans/reflection-implementation-plan.md +898 -0
- package/docs/target-ir-and-adapters.md +111 -0
- package/docs/validation-process.md +172 -0
- package/docs/visual-contract.md +174 -0
- package/package.json +62 -0
|
@@ -0,0 +1,898 @@
|
|
|
1
|
+
# Reflection Implementation Plan
|
|
2
|
+
|
|
3
|
+
> **For Hermes:** Use this as the working implementation guide. Keep Day 1 focused on the smallest useful rendered-evidence loop before expanding into visual baselines, design contracts, Storybook, or Figma parity.
|
|
4
|
+
|
|
5
|
+
**Goal:** Build Reflection as a separate sibling repo: a small CLI that produces evidence-backed validation of rendered UI behavior, screenshots, reports, and CI decisions.
|
|
6
|
+
|
|
7
|
+
**Product stance:** Reflection should not use Greenhouse in the public package, binary, repo, config helper, or user-facing language. It may later integrate with external spec systems through adapters, but the product identity is standalone: `reflection`.
|
|
8
|
+
|
|
9
|
+
**Architecture:** Start with a TypeScript CLI around a narrow core: config loading, run planning, browser route execution, artifact writing, report generation, and exit-code classification. Add visual baselines, design command wrapping, and Storybook/Figma parity only after the browser evidence loop is working and verified.
|
|
10
|
+
|
|
11
|
+
**Tech stack:** TypeScript, Node >=22, pnpm, Commander, Zod, Vitest, Playwright, later `pngjs` + `pixelmatch` for image diffs.
|
|
12
|
+
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
## Product principles to preserve
|
|
16
|
+
|
|
17
|
+
1. **Small surface, deep evidence.** The user sees `reflection run`, `reflection review`, `reflection update`, and `reflection doctor`; implementation depth stays underneath.
|
|
18
|
+
2. **Rendered evidence beats claims.** A successful run must produce real browser evidence, machine-readable results, and human-readable summaries.
|
|
19
|
+
3. **No magic healing.** Reflection must never silently update baselines, raise thresholds, ignore environment mismatches, or mask dynamic content without explicit config.
|
|
20
|
+
4. **Sensitive by default.** Screenshots, traces, logs, cookies, and network data can leak private information. Traces and verbose network logs stay off unless explicitly configured or needed on failure.
|
|
21
|
+
5. **Review-only visual first.** Visual diffs start as review items unless a specific stable case is explicitly configured as blocking or strict mode is requested.
|
|
22
|
+
6. **Adapters, not coupling.** Future spec-system integration should compile external route/component/design metadata into Reflection targets. Reflection should not require a specific upstream spec product.
|
|
23
|
+
7. **Agents are first-class consumers.** `report.json` and `--json` output must make the next action clear: fix code, inspect artifact, update baseline, or ask a human.
|
|
24
|
+
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
## Current repository setup
|
|
28
|
+
|
|
29
|
+
Target path:
|
|
30
|
+
|
|
31
|
+
```text
|
|
32
|
+
/opt/data/workspace/repos/reflection
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Public package identity:
|
|
36
|
+
|
|
37
|
+
```json
|
|
38
|
+
{
|
|
39
|
+
"name": "reflection-check",
|
|
40
|
+
"bin": {
|
|
41
|
+
"reflection": "./dist/cli.js",
|
|
42
|
+
"reflection-check": "./dist/cli.js"
|
|
43
|
+
},
|
|
44
|
+
"publishConfig": {
|
|
45
|
+
"access": "public"
|
|
46
|
+
}
|
|
47
|
+
}
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
Do not use scoped or product names containing `greenhouse` during this iteration.
|
|
51
|
+
|
|
52
|
+
---
|
|
53
|
+
|
|
54
|
+
## Planned repository shape
|
|
55
|
+
|
|
56
|
+
```text
|
|
57
|
+
reflection/
|
|
58
|
+
README.md
|
|
59
|
+
package.json
|
|
60
|
+
tsconfig.json
|
|
61
|
+
vitest.config.ts
|
|
62
|
+
docs/
|
|
63
|
+
plans/
|
|
64
|
+
reflection-implementation-plan.md
|
|
65
|
+
src/
|
|
66
|
+
cli.ts
|
|
67
|
+
commands/
|
|
68
|
+
run.ts
|
|
69
|
+
review.ts
|
|
70
|
+
update.ts
|
|
71
|
+
doctor.ts
|
|
72
|
+
gc.ts
|
|
73
|
+
core/
|
|
74
|
+
config.ts
|
|
75
|
+
define-reflection.ts
|
|
76
|
+
run-planner.ts
|
|
77
|
+
environment.ts
|
|
78
|
+
artifact-store.ts
|
|
79
|
+
manifest.ts
|
|
80
|
+
report-schema.ts
|
|
81
|
+
report-writer.ts
|
|
82
|
+
baseline-store.ts
|
|
83
|
+
failure-classifier.ts
|
|
84
|
+
exit-codes.ts
|
|
85
|
+
contracts/
|
|
86
|
+
browser/
|
|
87
|
+
browser-contract.ts
|
|
88
|
+
route-runner.ts
|
|
89
|
+
assertions.ts
|
|
90
|
+
console-observer.ts
|
|
91
|
+
overflow-check.ts
|
|
92
|
+
fixtures.ts
|
|
93
|
+
visual/
|
|
94
|
+
visual-contract.ts
|
|
95
|
+
image-diff.ts
|
|
96
|
+
baseline-compare.ts
|
|
97
|
+
masks.ts
|
|
98
|
+
design/
|
|
99
|
+
design-contract.ts
|
|
100
|
+
command-adapter.ts
|
|
101
|
+
integrations/
|
|
102
|
+
playwright/
|
|
103
|
+
browser-manager.ts
|
|
104
|
+
context-factory.ts
|
|
105
|
+
trace-policy.ts
|
|
106
|
+
storybook/
|
|
107
|
+
story-url.ts
|
|
108
|
+
index-json.ts
|
|
109
|
+
utils/
|
|
110
|
+
fs-safe.ts
|
|
111
|
+
hashing.ts
|
|
112
|
+
logger.ts
|
|
113
|
+
time.ts
|
|
114
|
+
examples/
|
|
115
|
+
basic-react/
|
|
116
|
+
package.json
|
|
117
|
+
src/
|
|
118
|
+
reflection.config.ts
|
|
119
|
+
tests/
|
|
120
|
+
unit/
|
|
121
|
+
integration/
|
|
122
|
+
e2e/
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
Only create files as needed. The shape above is the intended destination, not permission to scaffold empty abstractions.
|
|
126
|
+
|
|
127
|
+
---
|
|
128
|
+
|
|
129
|
+
## Day 1: Browser evidence loop
|
|
130
|
+
|
|
131
|
+
### Day 1 goal
|
|
132
|
+
|
|
133
|
+
`reflection run` validates at least one route in a real browser, captures screenshot evidence, writes report artifacts, and exits with the correct status.
|
|
134
|
+
|
|
135
|
+
### Day 1 non-goals
|
|
136
|
+
|
|
137
|
+
- No Figma API.
|
|
138
|
+
- No Storybook integration.
|
|
139
|
+
- No design-token validator wrapping.
|
|
140
|
+
- No baseline updates.
|
|
141
|
+
- No full HTML report viewer.
|
|
142
|
+
- No broad framework auto-detection.
|
|
143
|
+
- No public publishing setup.
|
|
144
|
+
|
|
145
|
+
### Phase 1.1 — CLI and config foundation
|
|
146
|
+
|
|
147
|
+
**Objective:** Make the CLI parse flags, load config, validate schema, and fail clearly.
|
|
148
|
+
|
|
149
|
+
**Files:**
|
|
150
|
+
- Modify: `src/cli.ts`
|
|
151
|
+
- Create: `src/commands/run.ts`
|
|
152
|
+
- Create: `src/commands/doctor.ts`
|
|
153
|
+
- Create: `src/core/config.ts`
|
|
154
|
+
- Create: `src/core/define-reflection.ts`
|
|
155
|
+
- Create: `src/core/exit-codes.ts`
|
|
156
|
+
- Test: `tests/unit/config.test.ts`
|
|
157
|
+
- Test: `tests/unit/cli.test.ts`
|
|
158
|
+
|
|
159
|
+
**Required behavior:**
|
|
160
|
+
|
|
161
|
+
```bash
|
|
162
|
+
reflection run --config examples/basic-react/reflection.config.ts
|
|
163
|
+
reflection run --mode smoke
|
|
164
|
+
reflection run --ci
|
|
165
|
+
reflection doctor
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
Config helper:
|
|
169
|
+
|
|
170
|
+
```ts
|
|
171
|
+
import { defineReflection } from 'reflection-check';
|
|
172
|
+
|
|
173
|
+
export default defineReflection({
|
|
174
|
+
project: 'basic-react',
|
|
175
|
+
run: {
|
|
176
|
+
defaultMode: 'smoke',
|
|
177
|
+
ciMode: 'smoke'
|
|
178
|
+
},
|
|
179
|
+
contracts: {
|
|
180
|
+
browser: {
|
|
181
|
+
enabled: true,
|
|
182
|
+
blocking: true,
|
|
183
|
+
baseUrl: 'http://127.0.0.1:5173',
|
|
184
|
+
routes: []
|
|
185
|
+
}
|
|
186
|
+
}
|
|
187
|
+
});
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
**Acceptance:**
|
|
191
|
+
|
|
192
|
+
- Missing config exits `2` with a useful message.
|
|
193
|
+
- Invalid CLI usage exits `64`.
|
|
194
|
+
- Valid minimal config loads and normalizes defaults.
|
|
195
|
+
- No user-facing package/config helper names contain `greenhouse`.
|
|
196
|
+
|
|
197
|
+
**Phase 1.1 evidence — 2026-05-31:**
|
|
198
|
+
|
|
199
|
+
- RED: `corepack pnpm exec vitest run tests/unit/config.test.ts tests/unit/cli.test.ts` failed because `src/core/config.ts`, `src/core/define-reflection.ts`, `src/core/exit-codes.ts`, and `createCli` did not exist yet; after first implementation pass it also exposed CLI error-handling issues for invalid mode and missing config.
|
|
200
|
+
- GREEN: Added config schema/default normalization, `defineReflection`, exit-code constants, command modules, and import-safe `createCli`.
|
|
201
|
+
- Verification: `corepack pnpm typecheck`, `corepack pnpm test`, `corepack pnpm build`, `node dist/cli.js run --mode smoke`, `node dist/cli.js doctor`, and `git diff --check` all passed.
|
|
202
|
+
- Current scope note: TypeScript config files are planned, but Phase 1.1 only proves runtime loading for JavaScript ESM config modules. True `reflection.config.ts` loading should be implemented deliberately when choosing the runtime loader strategy.
|
|
203
|
+
|
|
204
|
+
### Phase 1.2 — Artifact and report core
|
|
205
|
+
|
|
206
|
+
**Objective:** Every run writes a manifest and canonical reports before browser complexity expands.
|
|
207
|
+
|
|
208
|
+
**Files:**
|
|
209
|
+
- Create: `src/core/artifact-store.ts`
|
|
210
|
+
- Create: `src/core/manifest.ts`
|
|
211
|
+
- Create: `src/core/report-schema.ts`
|
|
212
|
+
- Create: `src/core/report-writer.ts`
|
|
213
|
+
- Create: `src/core/failure-classifier.ts`
|
|
214
|
+
- Test: `tests/unit/report-schema.test.ts`
|
|
215
|
+
- Test: `tests/unit/artifact-store.test.ts`
|
|
216
|
+
|
|
217
|
+
**Artifact layout:**
|
|
218
|
+
|
|
219
|
+
```text
|
|
220
|
+
.reflection/
|
|
221
|
+
runs/
|
|
222
|
+
<run-id>/
|
|
223
|
+
manifest.json
|
|
224
|
+
report.json
|
|
225
|
+
report.md
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
**Canonical result model:**
|
|
229
|
+
|
|
230
|
+
```ts
|
|
231
|
+
type CheckResult = {
|
|
232
|
+
id: string;
|
|
233
|
+
suite: 'design' | 'browser' | 'visual' | 'component' | 'environment';
|
|
234
|
+
target: string;
|
|
235
|
+
status: 'pass' | 'fail' | 'warn' | 'skipped' | 'error';
|
|
236
|
+
severity: 'blocking' | 'review' | 'info';
|
|
237
|
+
summary: string;
|
|
238
|
+
details?: string;
|
|
239
|
+
artifacts: ArtifactRef[];
|
|
240
|
+
metadata: Record<string, unknown>;
|
|
241
|
+
suggestedNextStep?: string;
|
|
242
|
+
};
|
|
243
|
+
```
|
|
244
|
+
|
|
245
|
+
**Acceptance:**
|
|
246
|
+
|
|
247
|
+
- Dummy pass/fail checks produce valid `report.json` and readable `report.md`.
|
|
248
|
+
- Global run status is derived from check status/severity.
|
|
249
|
+
- `pass + review warning` exits `0` but reports `pass-with-review`.
|
|
250
|
+
- Blocking failure exits `1`.
|
|
251
|
+
- Tool/config/internal error exits `2`.
|
|
252
|
+
|
|
253
|
+
**Phase 1.2 evidence — 2026-05-31:**
|
|
254
|
+
|
|
255
|
+
- RED: `corepack pnpm exec vitest run tests/unit/report-schema.test.ts tests/unit/artifact-store.test.ts` failed because `src/core/report-schema.ts`, `src/core/artifact-store.ts`, `src/core/manifest.ts`, and `src/core/report-writer.ts` did not exist yet.
|
|
256
|
+
- GREEN: Added canonical report schemas/types, status/exit-code derivation, artifact store with path traversal guard, manifest creator, Markdown/JSON report writer, and basic failure classifier.
|
|
257
|
+
- CLI smoke: `node dist/cli.js run --mode smoke --report-dir /tmp/reflection-phase-1-2-smoke` wrote `report.json`, `report.md`, `manifest.json`, and a `runs/latest` pointer.
|
|
258
|
+
- Verification: `corepack pnpm typecheck`, `corepack pnpm test`, `corepack pnpm build`, CLI smoke, and `git diff --check` passed.
|
|
259
|
+
- Current scope note: `reflection run` now emits a placeholder environment check so the evidence pipeline exists before real browser contract checks. Phase 1.3 should replace the placeholder context with a real fixture app target; Phase 1.5 should replace the placeholder check with Playwright-produced browser checks.
|
|
260
|
+
|
|
261
|
+
### Phase 1.3 — Example app fixture
|
|
262
|
+
|
|
263
|
+
**Objective:** Provide a tiny real app that can prove the browser runner.
|
|
264
|
+
|
|
265
|
+
**Files:**
|
|
266
|
+
- Create: `examples/basic-react/package.json`
|
|
267
|
+
- Create: `examples/basic-react/index.html`
|
|
268
|
+
- Create: `examples/basic-react/src/main.tsx`
|
|
269
|
+
- Create: `examples/basic-react/reflection.config.ts`
|
|
270
|
+
|
|
271
|
+
**Routes/states:**
|
|
272
|
+
|
|
273
|
+
```text
|
|
274
|
+
/login
|
|
275
|
+
desktop + mobile
|
|
276
|
+
heading exists
|
|
277
|
+
email/password labels exist
|
|
278
|
+
sign-in button exists
|
|
279
|
+
no signup/register text
|
|
280
|
+
no horizontal overflow
|
|
281
|
+
screenshot evidence
|
|
282
|
+
|
|
283
|
+
/overflow
|
|
284
|
+
mobile intentionally overflows
|
|
285
|
+
should fail noHorizontalOverflow
|
|
286
|
+
|
|
287
|
+
/console-error
|
|
288
|
+
intentionally logs console.error
|
|
289
|
+
should fail noConsoleErrors
|
|
290
|
+
```
|
|
291
|
+
|
|
292
|
+
**Acceptance:**
|
|
293
|
+
|
|
294
|
+
- The fixture can run locally with Vite.
|
|
295
|
+
- It has deterministic content, no dynamic timestamps, no network dependency.
|
|
296
|
+
- It provides one passing route and two failing route cases for tests.
|
|
297
|
+
|
|
298
|
+
**Phase 1.3 evidence — 2026-05-31:**
|
|
299
|
+
|
|
300
|
+
- RED: `corepack pnpm exec vitest run tests/unit/basic-react-fixture.test.ts` failed because `examples/basic-react/package.json`, `src/main.tsx`, and `reflection.config.ts` did not exist yet.
|
|
301
|
+
- GREEN: Added a minimal Vite/React fixture with deterministic `/login`, intentionally overflowing `/overflow`, and intentional console-error `/console-error` routes plus a Reflection browser contract config for those scenarios.
|
|
302
|
+
- Fixture smoke: `corepack pnpm install` and `corepack pnpm build` passed in `examples/basic-react`; a Vite dev server returned HTTP 200 for `/login`, `/overflow`, and `/console-error`.
|
|
303
|
+
- Verification: `corepack pnpm typecheck`, `corepack pnpm test`, `corepack pnpm build`, and `git diff --check` passed at the Reflection repo root.
|
|
304
|
+
- Current scope note: the fixture includes `reflection.config.ts` for the intended typed public surface, but runtime TS config loading is still a deliberate future task. Phase 1.4 can use the fixture server command/readiness values without needing TS config execution.
|
|
305
|
+
|
|
306
|
+
### Phase 1.4 — Server manager
|
|
307
|
+
|
|
308
|
+
**Objective:** Start or reuse a configured app server and wait for readiness.
|
|
309
|
+
|
|
310
|
+
**Files:**
|
|
311
|
+
- Create: `src/core/server-manager.ts`
|
|
312
|
+
- Create: `src/utils/process.ts`
|
|
313
|
+
- Test: `tests/integration/server-manager.test.ts`
|
|
314
|
+
|
|
315
|
+
**Required behavior:**
|
|
316
|
+
|
|
317
|
+
```ts
|
|
318
|
+
server: {
|
|
319
|
+
command: 'pnpm dev --host 127.0.0.1',
|
|
320
|
+
readyUrl: 'http://127.0.0.1:5173',
|
|
321
|
+
reuseExisting: true,
|
|
322
|
+
timeoutMs: 60_000
|
|
323
|
+
}
|
|
324
|
+
```
|
|
325
|
+
|
|
326
|
+
**Acceptance:**
|
|
327
|
+
|
|
328
|
+
- Reuses an already reachable `readyUrl` when configured.
|
|
329
|
+
- Starts the server when needed.
|
|
330
|
+
- Captures server logs into run artifacts on failure.
|
|
331
|
+
- Kills only processes it started.
|
|
332
|
+
|
|
333
|
+
**Phase 1.4 evidence — 2026-05-31:**
|
|
334
|
+
|
|
335
|
+
- RED: `corepack pnpm exec vitest run tests/integration/server-manager.test.ts` failed because `src/core/server-manager.ts` did not exist yet.
|
|
336
|
+
- GREEN: Added `startManagedServer`, `waitForUrl`, and a small managed process utility that starts shell commands, captures stdout/stderr to a configured log path, reuses reachable servers, and terminates only owned process groups.
|
|
337
|
+
- Focused verification: `corepack pnpm exec vitest run tests/integration/server-manager.test.ts` passed with reuse, start/wait/logging, and timeout cleanup coverage.
|
|
338
|
+
- Fixture smoke: built `dist`, then started `examples/basic-react` through the built server manager; it reported `{ started: true, reused: false, pidType: "number" }`, `/overflow` became reachable, and `stop()` made `/login` unreachable again.
|
|
339
|
+
- Verification: `corepack pnpm typecheck`, `corepack pnpm test`, `corepack pnpm build`, and `git diff --check` passed.
|
|
340
|
+
- Current scope note: the server manager is not wired into `reflection run` yet. Phase 1.5 should use it as the substrate for Playwright browser route execution and route artifact logs.
|
|
341
|
+
|
|
342
|
+
### Phase 1.5 — Playwright browser route runner
|
|
343
|
+
|
|
344
|
+
**Objective:** Render configured routes in Chromium across viewports and fixture states.
|
|
345
|
+
|
|
346
|
+
**Files:**
|
|
347
|
+
- Create: `src/integrations/playwright/browser-manager.ts`
|
|
348
|
+
- Create: `src/integrations/playwright/context-factory.ts`
|
|
349
|
+
- Create: `src/contracts/browser/browser-contract.ts`
|
|
350
|
+
- Create: `src/contracts/browser/route-runner.ts`
|
|
351
|
+
- Create: `src/contracts/browser/console-observer.ts`
|
|
352
|
+
- Create: `src/contracts/browser/overflow-check.ts`
|
|
353
|
+
- Create: `src/contracts/browser/assertions.ts`
|
|
354
|
+
- Test: `tests/integration/browser-contract.test.ts`
|
|
355
|
+
|
|
356
|
+
**MVP assertions:**
|
|
357
|
+
|
|
358
|
+
```text
|
|
359
|
+
urlIncludes
|
|
360
|
+
urlEquals
|
|
361
|
+
role
|
|
362
|
+
label
|
|
363
|
+
text
|
|
364
|
+
noText
|
|
365
|
+
selector
|
|
366
|
+
elementVisible
|
|
367
|
+
elementNotVisible
|
|
368
|
+
noHorizontalOverflow
|
|
369
|
+
noConsoleErrors
|
|
370
|
+
screenshot
|
|
371
|
+
```
|
|
372
|
+
|
|
373
|
+
**Screenshot output:**
|
|
374
|
+
|
|
375
|
+
```text
|
|
376
|
+
.reflection/runs/<run-id>/browser/<route-id>/<viewport>/actual.png
|
|
377
|
+
.reflection/runs/<run-id>/browser/<route-id>/<viewport>/metadata.json
|
|
378
|
+
```
|
|
379
|
+
|
|
380
|
+
**Acceptance:**
|
|
381
|
+
|
|
382
|
+
- `/login` passes on desktop and mobile.
|
|
383
|
+
- `/overflow` fails on mobile with a clear `layout-overflow` classification.
|
|
384
|
+
- `/console-error` fails with a clear `console-error` classification.
|
|
385
|
+
- Browser failures become blocking failures by default.
|
|
386
|
+
- Screenshots are saved and referenced in `report.json` and `report.md`.
|
|
387
|
+
|
|
388
|
+
**Phase 1.5 evidence — 2026-05-31:**
|
|
389
|
+
|
|
390
|
+
- RED: `corepack pnpm exec vitest run tests/integration/browser-contract.test.ts` failed because `src/contracts/browser/browser-contract.js` did not exist yet.
|
|
391
|
+
- GREEN: Added Playwright Chromium launch/context helpers, route runner, browser assertions, console-error observer, horizontal-overflow check, screenshot/metadata artifacts, and browser contract aggregation. Added `playwright` for browser execution.
|
|
392
|
+
- Fixture correction: `/login` mobile initially failed `noHorizontalOverflow`; fixed the fixture panel to use `boxSizing: 'border-box'` so the passing route is genuinely stable on mobile.
|
|
393
|
+
- CLI wiring: `reflection run` now starts the configured server, runs browser checks for `smoke`/`full`, writes screenshot artifacts into `browser/<route>/<viewport>/actual.png`, writes per-route metadata, summarizes check statuses, and exits `1` on blocking browser failures.
|
|
394
|
+
- Focused verification: `corepack pnpm exec vitest run tests/integration/browser-contract.test.ts` passed; `/login` passed desktop/mobile, `/overflow` failed as `layout-overflow`, and `/console-error` failed as `console-error`.
|
|
395
|
+
- CLI smoke: a login-only fixture config exited `0` with status `pass` and 2 screenshots; an all-routes fixture config exited `1` with status `fail`, 2 blocking failures, and 3 screenshots.
|
|
396
|
+
- Verification: `corepack pnpm typecheck`, `corepack pnpm test`, `corepack pnpm build`, and `git diff --check` passed.
|
|
397
|
+
- Current scope note: runtime loading of `reflection.config.ts` is still deferred. CLI smoke used temporary `.mjs` configs; Phase 1.6 should decide whether to add TS config loading now or document/use JS configs until the loader strategy is explicit.
|
|
398
|
+
|
|
399
|
+
### Phase 1.6 — Day 1 verification gate
|
|
400
|
+
|
|
401
|
+
**Objective:** Prove Day 1 works end-to-end.
|
|
402
|
+
|
|
403
|
+
**Commands:**
|
|
404
|
+
|
|
405
|
+
```bash
|
|
406
|
+
pnpm install
|
|
407
|
+
pnpm typecheck
|
|
408
|
+
pnpm test
|
|
409
|
+
pnpm build
|
|
410
|
+
pnpm reflection run --config examples/basic-react/reflection.config.ts
|
|
411
|
+
pnpm reflection doctor --config examples/basic-react/reflection.config.ts
|
|
412
|
+
```
|
|
413
|
+
|
|
414
|
+
**Acceptance:**
|
|
415
|
+
|
|
416
|
+
- All tests pass.
|
|
417
|
+
- Build passes.
|
|
418
|
+
- Passing fixture route produces artifacts and exit code `0` when only passing routes are selected.
|
|
419
|
+
- Failing fixture cases produce exit code `1` and clear report entries.
|
|
420
|
+
- Report paths are real and readable.
|
|
421
|
+
- Final `git status --short` is reviewed before any commit.
|
|
422
|
+
|
|
423
|
+
**Phase 1.6 evidence — 2026-05-31:**
|
|
424
|
+
|
|
425
|
+
- Decision: baked in runtime `reflection.config.ts` loading now instead of continuing with temporary JavaScript configs, using `jiti` as the smallest practical loader for TypeScript config files.
|
|
426
|
+
- RED: `node dist/cli.js run --config examples/basic-react/reflection.config.ts --report-dir <tmp>` exited `2` because the built CLI could not import the TypeScript config/helper path.
|
|
427
|
+
- GREEN: `loadReflectionConfig` now resolves config paths to absolute paths and uses `jiti` for `.ts`, `.mts`, and `.cts` modules while preserving native dynamic import for JavaScript modules. Added regression coverage for TypeScript config files and relative TypeScript config paths.
|
|
428
|
+
- Fixture adjustment: the example config server command now uses `pnpm --dir examples/basic-react dev --host 127.0.0.1` so the built CLI can start the fixture from the repo root.
|
|
429
|
+
- CLI smoke pass case: a temporary TypeScript login-only config exited `0`, reported status `pass`, project `basic-react-login-only`, and wrote desktop/mobile screenshot evidence.
|
|
430
|
+
- CLI smoke fail case: `node dist/cli.js run --config examples/basic-react/reflection.config.ts --report-dir <tmp>` exited `1`, reported status `fail`, project `basic-react`, 2 blocking failures, and classified them as `layout-overflow` and `console-error`.
|
|
431
|
+
- Verification: `corepack pnpm typecheck`, `corepack pnpm test` (8 files / 30 tests), `corepack pnpm build`, and `git diff --check` passed.
|
|
432
|
+
|
|
433
|
+
---
|
|
434
|
+
|
|
435
|
+
## Day 2: Visual smoke seam and review flow
|
|
436
|
+
|
|
437
|
+
### Day 2 goal
|
|
438
|
+
|
|
439
|
+
Add the smallest visual baseline comparison loop without making baselines dangerous.
|
|
440
|
+
|
|
441
|
+
### DevEx installation/setup direction
|
|
442
|
+
|
|
443
|
+
Reflection should have friendly setup, but it should not start with a magical broad `init` that silently mutates a repo. The better shape is a staged setup flow:
|
|
444
|
+
|
|
445
|
+
1. `reflection doctor` is read-only and always safe. It explains what is missing and can print suggested config/scripts.
|
|
446
|
+
2. `reflection init --dry-run` detects package manager/framework/dev server/Storybook and prints the exact files and package scripts it would create.
|
|
447
|
+
3. `reflection init --write` creates only the minimal files: `reflection.config.ts`, `.reflection/.gitkeep`, and suggested package scripts.
|
|
448
|
+
4. `reflection init --preset vite-react` / `--preset storybook` can be added before broad auto-detection, so the first experience is deterministic rather than clever.
|
|
449
|
+
5. `reflection init --from-greenhouse` can later adapt Greenhouse Spec route/component metadata into Reflection config without coupling the product identity to Greenhouse.
|
|
450
|
+
|
|
451
|
+
Recommendation: build the first install path as **doctor-first + dry-run init**, then add explicit presets. This keeps the nice Greenhouse-init feeling while preserving Reflection's safety rule: no silent baselines, no silent thresholds, no broad repo mutation.
|
|
452
|
+
|
|
453
|
+
### Phase 2.1 — Baseline store
|
|
454
|
+
|
|
455
|
+
**Objective:** Load baseline metadata and enforce safe baseline path rules.
|
|
456
|
+
|
|
457
|
+
**Files:**
|
|
458
|
+
- Create: `src/core/baseline-store.ts`
|
|
459
|
+
- Test: `tests/unit/baseline-store.test.ts`
|
|
460
|
+
|
|
461
|
+
**Acceptance:**
|
|
462
|
+
|
|
463
|
+
- Baselines resolve only inside `.reflection/baselines/` or configured baseline root.
|
|
464
|
+
- Missing baseline produces a controlled review/fail result according to case policy.
|
|
465
|
+
- Normal runs never create or update baselines.
|
|
466
|
+
|
|
467
|
+
**Phase 2.1 evidence — 2026-05-31:**
|
|
468
|
+
|
|
469
|
+
- RED: `corepack pnpm exec vitest run tests/unit/baseline-store.test.ts` failed because `src/core/baseline-store.ts` did not exist yet.
|
|
470
|
+
- GREEN: Added a read-only baseline store that resolves paths only inside the configured baseline root, reads metadata, and creates controlled missing-baseline visual checks as review warnings or blocking failures based on policy.
|
|
471
|
+
- Safety note: the baseline store intentionally exposes no write/update API; baseline mutation remains reserved for the later explicit `reflection update` flow.
|
|
472
|
+
- Verification: `corepack pnpm exec vitest run tests/unit/baseline-store.test.ts`, `corepack pnpm typecheck`, `corepack pnpm test` (9 files / 33 tests), `corepack pnpm build`, and `git diff --check` passed.
|
|
473
|
+
|
|
474
|
+
### Phase 2.2 — Image diff service
|
|
475
|
+
|
|
476
|
+
**Objective:** Compare actual and expected PNGs and produce metadata/diff image.
|
|
477
|
+
|
|
478
|
+
**Files:**
|
|
479
|
+
- Create: `src/contracts/visual/image-diff.ts`
|
|
480
|
+
- Create: `src/contracts/visual/thresholds.ts`
|
|
481
|
+
- Test: `tests/unit/image-diff.test.ts`
|
|
482
|
+
|
|
483
|
+
**Dependencies:**
|
|
484
|
+
|
|
485
|
+
```bash
|
|
486
|
+
pnpm add pngjs pixelmatch
|
|
487
|
+
pnpm add -D @types/pngjs
|
|
488
|
+
```
|
|
489
|
+
|
|
490
|
+
**Acceptance:**
|
|
491
|
+
|
|
492
|
+
- Equal PNGs pass.
|
|
493
|
+
- Dimension mismatch is classified separately.
|
|
494
|
+
- Diff over threshold becomes `warn` by default.
|
|
495
|
+
- Strict mode can convert selected diffs into blocking failure.
|
|
496
|
+
|
|
497
|
+
**Phase 2.2 evidence — 2026-05-31:**
|
|
498
|
+
|
|
499
|
+
- RED: `corepack pnpm exec vitest run tests/unit/image-diff.test.ts` failed because `src/contracts/visual/image-diff.ts` did not exist yet.
|
|
500
|
+
- GREEN: Added `pngjs`, `pixelmatch`, `@types/pngjs`, a pure PNG comparison service, and threshold evaluation for `maxDiffPixels` / `maxDiffPixelRatio`.
|
|
501
|
+
- Behavior: equal PNGs pass and can write a diff artifact, dimension mismatches classify as `visual-dimension-mismatch`, over-threshold diffs warn by default, and strict mode converts over-threshold diffs to fail.
|
|
502
|
+
- Verification: `corepack pnpm exec vitest run tests/unit/image-diff.test.ts`, `corepack pnpm typecheck`, `corepack pnpm test` (10 files / 38 tests), `corepack pnpm build`, and `git diff --check` passed.
|
|
503
|
+
|
|
504
|
+
### Phase 2.3 — Route visual smoke
|
|
505
|
+
|
|
506
|
+
**Objective:** Compare one route screenshot against a baseline.
|
|
507
|
+
|
|
508
|
+
**Files:**
|
|
509
|
+
- Create: `src/contracts/visual/visual-contract.ts`
|
|
510
|
+
- Create: `src/contracts/visual/baseline-compare.ts`
|
|
511
|
+
- Modify: `src/contracts/browser/route-runner.ts`
|
|
512
|
+
- Test: `tests/integration/visual-smoke.test.ts`
|
|
513
|
+
|
|
514
|
+
**Acceptance:**
|
|
515
|
+
|
|
516
|
+
- `/login` mobile actual screenshot compares to a checked-in fixture baseline.
|
|
517
|
+
- Actual/baseline/diff files are linked in the report.
|
|
518
|
+
- Visual diff is review-only unless strict mode is set.
|
|
519
|
+
|
|
520
|
+
**Phase 2.3 evidence — 2026-05-31:**
|
|
521
|
+
|
|
522
|
+
- RED: `corepack pnpm exec vitest run tests/integration/visual-smoke.test.ts` failed because `runBrowserContract` only returned browser checks; `visual.login-mobile` and missing-baseline visual checks were absent.
|
|
523
|
+
- GREEN: Added route visual smoke cases, read-only baseline resolution, visual expected/actual/diff artifact linking, and review-only missing-baseline reporting. The basic React fixture now has a checked-in `/login` mobile baseline.
|
|
524
|
+
- CLI smoke: `node dist/cli.js run --config /tmp/reflection-visual-smoke.config.mjs --report-dir /tmp/reflection-visual-cli-ioM2BC` passed with `environment.smoke.server`, `browser.login.mobile`, and `visual.login-mobile`; `report.json` linked `visual/login-mobile/{expected,actual,diff}.png` plus the browser actual screenshot.
|
|
525
|
+
- Verification: `corepack pnpm exec vitest run tests/integration/visual-smoke.test.ts`, `corepack pnpm typecheck`, `corepack pnpm test` (11 files / 40 tests), `corepack pnpm build`, and `git diff --check` passed.
|
|
526
|
+
|
|
527
|
+
### Phase 2.4 — Review command
|
|
528
|
+
|
|
529
|
+
**Objective:** Make latest evidence easy to inspect from the CLI.
|
|
530
|
+
|
|
531
|
+
**Files:**
|
|
532
|
+
- Create: `src/commands/review.ts`
|
|
533
|
+
- Test: `tests/unit/review-command.test.ts`
|
|
534
|
+
|
|
535
|
+
**Acceptance:**
|
|
536
|
+
|
|
537
|
+
```bash
|
|
538
|
+
reflection review
|
|
539
|
+
reflection review --latest
|
|
540
|
+
reflection review --json
|
|
541
|
+
```
|
|
542
|
+
|
|
543
|
+
prints:
|
|
544
|
+
|
|
545
|
+
- status
|
|
546
|
+
- blocking failures
|
|
547
|
+
- review items
|
|
548
|
+
- artifact paths
|
|
549
|
+
- suggested next steps
|
|
550
|
+
|
|
551
|
+
**Phase 2.4 evidence — 2026-05-31:**
|
|
552
|
+
|
|
553
|
+
- Validation phase before new work: independent review of Phase 2.3 found no secret/shell/eval/deserialization/SQL concerns, but did find two logic issues: dimension-mismatch visual comparisons could crash when no diff image was written, and `blocking: true` visual smoke cases were still reported as review warnings unless `strict` was also set.
|
|
554
|
+
- Fixes from validation: added unit coverage for `compareRouteVisualBaseline`; dimension mismatches now return a visual check with expected/actual artifacts and no missing diff stat; blocking visual diffs now become `status: fail` + `severity: blocking`; visual `maxDiffPixelRatio` config is constrained to `0..1`.
|
|
555
|
+
- RED: `corepack pnpm exec vitest run tests/unit/review-command.test.ts` initially failed because `src/commands/review.ts` did not exist. Config and baseline-compare validation tests also failed before their fixes.
|
|
556
|
+
- GREEN: Added `reflection review` CLI wiring plus `src/commands/review.ts`. The command reads `runs/latest` or `--run`, validates run IDs/path boundaries, parses `report.json`, prints human review output, and emits a stable JSON agent summary with blocking failures, review items, artifact paths, and next steps.
|
|
557
|
+
- Review hardening: rejects ambiguous `--latest` + `--run`, rejects unsafe latest-pointer run IDs, and has CLI-level coverage for `reflection review --json --report-dir <dir> --run <id>`.
|
|
558
|
+
- CLI smoke: `node dist/cli.js run --config /tmp/reflection-review-smoke.config.mjs --report-dir /tmp/reflection-review-cli-Z4VMot`, then `node dist/cli.js review --report-dir /tmp/reflection-review-cli-Z4VMot`, then `node dist/cli.js review --report-dir /tmp/reflection-review-cli-Z4VMot --json` passed. The review output listed `browser/login/mobile/actual.png`, metadata, and `visual/login-mobile/{expected,actual,diff}.png`.
|
|
559
|
+
- Verification: `corepack pnpm exec vitest run tests/unit/review-command.test.ts`, `corepack pnpm test` (13 files / 49 tests), `corepack pnpm typecheck`, `corepack pnpm build`, and `git diff --check` passed.
|
|
560
|
+
- Final independent validation review passed with no blocking security concerns or logic errors. Follow-up suggestions kept for later: include summary counts/passed checks in human review output if needed.
|
|
561
|
+
|
|
562
|
+
### Phase 2.5 — Update command dry-run and targeted update
|
|
563
|
+
|
|
564
|
+
**Objective:** Add explicit baseline update flow without CI mutation risk.
|
|
565
|
+
|
|
566
|
+
**Files:**
|
|
567
|
+
- Create: `src/commands/update.ts`
|
|
568
|
+
- Test: `tests/integration/update-command.test.ts`
|
|
569
|
+
|
|
570
|
+
**Acceptance:**
|
|
571
|
+
|
|
572
|
+
- `reflection update --route login --from-run latest --dry-run` reports intended changes without writing.
|
|
573
|
+
- Non-dry targeted update copies only the selected actual screenshot into the selected baseline path.
|
|
574
|
+
- CI mode refuses baseline updates.
|
|
575
|
+
- Untargeted “update everything” is avoided or requires an explicit `--all` guard.
|
|
576
|
+
|
|
577
|
+
**Phase 2.5 evidence — 2026-06-01:**
|
|
578
|
+
|
|
579
|
+
- Validation phase before new work: independent review of Phase 2.4 found no secret/shell/eval/deserialization/SQL concerns, but requested hardening for report symlink reads and unsafe artifact paths. `reflection review` now realpath-checks `runs/latest`, the selected run directory, and `report.json`; rejects report/run mismatches; and rejects absolute or escaping artifact paths before emitting agent summaries.
|
|
580
|
+
- RED: `corepack pnpm exec vitest run tests/integration/update-command.test.ts` initially failed because `src/commands/update.ts` did not exist. CLI wiring coverage then failed before `src/cli.ts` registered `update`.
|
|
581
|
+
- GREEN: Added `reflection update` CLI wiring plus `src/commands/update.ts`. The command supports `--route`, `--case`, explicit `--all`, `--from-run latest|<runId>`, and `--dry-run`; non-dry updates copy only selected actual PNG artifacts into configured baseline paths.
|
|
582
|
+
- Safety hardening: update refuses CI mutation, untargeted updates, mixed `--all` + targeted selectors, unsafe run IDs, symlinked/escaping `latest`, run directories, report files, and source artifacts. Baseline writes validate the configured root and walk/create destination directories without following intermediate symlinks; existing destination symlinks are refused before overwrite.
|
|
583
|
+
- RED/GREEN validation loop: regression tests first failed for `report.json` symlink escape, mixed `--all` selectors, symlinked `runs/latest`, review latest symlink escape, and intermediate baseline directory symlink creation outside the baseline root; each passed after the targeted fix.
|
|
584
|
+
- Verification: `corepack pnpm exec vitest run tests/unit/review-command.test.ts tests/integration/update-command.test.ts` (21 tests), `corepack pnpm typecheck`, `corepack pnpm test` (14 files / 66 tests), `corepack pnpm build`, and `git diff --check` passed.
|
|
585
|
+
- Final independent validation review passed with no blocking security concerns or logic errors after the symlink/path hardening fixes.
|
|
586
|
+
|
|
587
|
+
---
|
|
588
|
+
|
|
589
|
+
## Day 3: Artifact lifecycle, safety, and CI shape
|
|
590
|
+
|
|
591
|
+
### Day 3 goal
|
|
592
|
+
|
|
593
|
+
Make generated evidence safe, maintainable, and CI-compatible.
|
|
594
|
+
|
|
595
|
+
### Phase 3.1 — Manifest-based GC
|
|
596
|
+
|
|
597
|
+
**Objective:** Delete only eligible run artifacts and never baselines.
|
|
598
|
+
|
|
599
|
+
**Files:**
|
|
600
|
+
- Create: `src/commands/gc.ts`
|
|
601
|
+
- Create: `src/core/gc.ts`
|
|
602
|
+
- Test: `tests/unit/gc.test.ts`
|
|
603
|
+
|
|
604
|
+
**Acceptance:**
|
|
605
|
+
|
|
606
|
+
- `reflection gc --dry-run` lists eligible run dirs.
|
|
607
|
+
- Deletion only occurs under configured artifact root.
|
|
608
|
+
- A directory must contain a valid manifest before deletion.
|
|
609
|
+
- Pinned runs are preserved.
|
|
610
|
+
- `.reflection/baselines/` is never deleted by normal GC.
|
|
611
|
+
|
|
612
|
+
**Phase 3.1 evidence — 2026-06-01:**
|
|
613
|
+
|
|
614
|
+
- RED: `corepack pnpm exec vitest run tests/unit/gc.test.ts` failed because `src/core/gc.ts` did not exist yet.
|
|
615
|
+
- GREEN: Added manifest-based `collectGarbage` plus `reflection gc` CLI wiring. GC dry-runs by default, supports explicit `--delete`, considers only run directories with valid manifests eligible, skips pinned runs, and reports skipped invalid/no-manifest entries.
|
|
616
|
+
- Safety hardening: GC rejects a symlinked `runs` directory, resolves candidates under the real configured runs directory, refuses/safely skips symlinked run directories that resolve outside the runs directory, revalidates candidate safety immediately before deletion, requires `manifest.runId` to match the directory name, ignores the `runs/latest` pointer, and never traverses into `.reflection/baselines/`.
|
|
617
|
+
- Review loop: independent pre-commit review found that validating candidates only against the artifact root could allow a symlinked `runs` directory to redirect deletion into in-root baselines. Added a RED regression for that case, then fixed GC to reject symlinked `runs` directories and revalidated.
|
|
618
|
+
- CLI smoke: built `dist`, created a temporary artifact root with one eligible run, one pinned run, and a baseline file; `node dist/cli.js gc --report-dir <tmp> --dry-run` listed only the eligible run, and `node dist/cli.js gc --report-dir <tmp> --delete` removed that run while preserving the pinned run and baseline.
|
|
619
|
+
- Verification: `corepack pnpm exec vitest run tests/unit/gc.test.ts`, `corepack pnpm typecheck`, `corepack pnpm test` (15 files / 71 tests), `corepack pnpm build`, CLI smoke, and `git diff --check` passed.
|
|
620
|
+
|
|
621
|
+
### Phase 3.2 — Redaction and artifact policy
|
|
622
|
+
|
|
623
|
+
**Objective:** Keep sensitive data out of reports/artifacts where possible.
|
|
624
|
+
|
|
625
|
+
**Files:**
|
|
626
|
+
- Create: `src/core/redaction.ts`
|
|
627
|
+
- Create: `src/integrations/playwright/trace-policy.ts`
|
|
628
|
+
- Test: `tests/unit/redaction.test.ts`
|
|
629
|
+
|
|
630
|
+
**Acceptance:**
|
|
631
|
+
|
|
632
|
+
- Authorization/cookie headers are redacted from captured logs.
|
|
633
|
+
- Configured `maskSelectors` are applied before screenshots.
|
|
634
|
+
- Report warns when traces/screenshots may contain private data.
|
|
635
|
+
- Videos are off by default.
|
|
636
|
+
|
|
637
|
+
**Phase 3.2 evidence — 2026-06-01:**
|
|
638
|
+
|
|
639
|
+
- RED: `corepack pnpm exec vitest run tests/unit/redaction.test.ts tests/unit/config.test.ts tests/integration/server-manager.test.ts tests/integration/browser-contract.test.ts` failed because `src/core/redaction.ts` / `src/integrations/playwright/trace-policy.ts` did not exist yet, `maskSelectors` were not accepted/preserved by config, server logs persisted auth/cookie values, and browser check metadata did not expose screenshot masking/privacy warnings.
|
|
640
|
+
- GREEN: Added `redactSensitiveText` and a streaming-safe redaction transform for managed server stdout/stderr logs; added browser `maskSelectors` config, applied matching selectors before screenshots, and recorded privacy warning/mask metadata in browser metadata/report checks; added a trace policy helper with trace/video capture off by default.
|
|
641
|
+
- Review hardening: independent review found that per-chunk redaction could leak sensitive values split across stream chunks and that JSON-serialized header fields were not covered. Added RED regressions for JSON auth/cookie/API key fields and chunk-boundary splits, then fixed the transform to buffer pending line/tail content before flushing and expanded quoted-field redaction.
|
|
642
|
+
- Re-review hardening: independent re-review found that bounded flushing could still leak long unterminated sensitive lines after the retained tail. Added a RED regression for long sensitive lines, then fixed the transform to suppress the remainder of an overlong sensitive line until a line terminator is observed.
|
|
643
|
+
- Final review hardening: independent final review found that escaped quotes inside JSON string values could end quoted-field redaction too early. Added a RED regression for escaped quotes in serialized sensitive values, then made quoted-value redaction escape-aware.
|
|
644
|
+
- CLI smoke: built `dist`, ran `node dist/cli.js run --config /tmp/reflection-phase-3-2.config.mjs --mode smoke --report-dir <tmp>` against the basic React login route with `maskSelectors: ['input[type="password"]']`; run passed, wrote screenshot artifacts, and `report.json` / `metadata.json` recorded the privacy warning plus applied mask selectors.
|
|
645
|
+
- Verification: focused Phase 3.2 tests, `corepack pnpm typecheck`, `corepack pnpm test` (16 files / 79 tests), `corepack pnpm build`, CLI smoke, and `git diff --check` passed.
|
|
646
|
+
|
|
647
|
+
### Phase 3.3 — CI report directory and workflow example
|
|
648
|
+
|
|
649
|
+
**Objective:** Make `reflection run --ci` reliable in CI.
|
|
650
|
+
|
|
651
|
+
**Files:**
|
|
652
|
+
- Create: `docs/ci.md`
|
|
653
|
+
- Create: `.github/workflows/reflection.yml` only when this repo has useful self-check examples
|
|
654
|
+
- Test: `tests/e2e/ci-mode.test.ts`
|
|
655
|
+
|
|
656
|
+
**Acceptance:**
|
|
657
|
+
|
|
658
|
+
- `--ci` writes to `artifacts/reflection` by default.
|
|
659
|
+
- workers default to `1` for visual stability.
|
|
660
|
+
- CI never updates baselines.
|
|
661
|
+
- Exit codes match the public spec.
|
|
662
|
+
|
|
663
|
+
**Phase 3.3 evidence — 2026-06-01:**
|
|
664
|
+
|
|
665
|
+
- RED: `corepack pnpm exec vitest run tests/e2e/ci-mode.test.ts` failed because CI reports did not record a stable `workers: 1` default and `docs/ci.md` / `.github/workflows/reflection.yml` did not exist yet.
|
|
666
|
+
- GREEN: Added CI worker default metadata for `reflection run --ci`, strict positive-integer `--workers` validation, documented CI artifact paths, baseline immutability, and public exit codes in `docs/ci.md`, and added a GitHub Actions workflow example that runs `node dist/cli.js run --ci --mode design` and uploads `artifacts/reflection`.
|
|
667
|
+
- Review hardening: independent review found malformed worker values such as `--workers 2abc` were coerced by `Number.parseInt`. Added a RED regression, then switched worker parsing to strict integer-string validation before artifact store creation.
|
|
668
|
+
- Re-review hardening: independent re-review found missing worker values (`--workers` without a count) still exited through Commander's default code `1`. Added a RED regression and changed `--workers` to an optional-value option whose missing value is validated by Reflection, returning exit code `64`.
|
|
669
|
+
- CLI smoke: built `dist`, ran `node /opt/data/workspace/repos/reflection/dist/cli.js run --ci --mode design` from a fresh temp directory, then read back `artifacts/reflection/runs/latest` and `report.json`; the report was `pass`, `ci: true`, profile `ci`, workers `1`, and `report.md` existed. Also smoke-checked `--workers 2abc` and missing `--workers` both exit `64` with invalid-workers messages.
|
|
670
|
+
- Verification: focused CI/update/CLI tests, `corepack pnpm typecheck`, `corepack pnpm test` (17 files / 85 tests), `corepack pnpm build`, CLI smoke, and `git diff --check` passed.
|
|
671
|
+
|
|
672
|
+
---
|
|
673
|
+
|
|
674
|
+
## Day 4: Design command adapter
|
|
675
|
+
|
|
676
|
+
### Day 4 goal
|
|
677
|
+
|
|
678
|
+
Wrap existing deterministic design/source validators without claiming pixel parity.
|
|
679
|
+
|
|
680
|
+
### Phase 4.1 — Command adapter
|
|
681
|
+
|
|
682
|
+
**Objective:** Run configured commands and normalize stdout/stderr/exit code into `CheckResult` values.
|
|
683
|
+
|
|
684
|
+
**Files:**
|
|
685
|
+
- Create: `src/contracts/design/design-contract.ts`
|
|
686
|
+
- Create: `src/contracts/design/command-adapter.ts`
|
|
687
|
+
- Test: `tests/integration/design-command-adapter.test.ts`
|
|
688
|
+
|
|
689
|
+
**Acceptance:**
|
|
690
|
+
|
|
691
|
+
- `reflection run --mode design` can run configured commands.
|
|
692
|
+
- Exit code `0` becomes pass.
|
|
693
|
+
- Non-zero exit becomes blocking failure unless configured otherwise.
|
|
694
|
+
- Output is summarized, with full logs as artifacts.
|
|
695
|
+
- The report wording says token/source contract, not full visual parity.
|
|
696
|
+
|
|
697
|
+
**Phase 4.1 evidence — 2026-06-01:**
|
|
698
|
+
|
|
699
|
+
- Resumption boundary: working tree already contained the Phase 4.1 RED/GREEN slice; inspected the plan, diff, tests, config schema, run-command wiring, and new design contract files before editing further.
|
|
700
|
+
- Focused GREEN boundary: `corepack pnpm exec vitest run tests/integration/design-command-adapter.test.ts && corepack pnpm typecheck` passed with 5 design adapter tests.
|
|
701
|
+
- Implementation: added typed design command config parsing, `runDesignContract`, `runDesignCommand`, design-mode run wiring, blocking/review-only normalization, token/source contract wording, and full stdout/stderr log artifacts.
|
|
702
|
+
- Review hardening: independent review passed with no blocking concerns and suggested direct CLI coverage for non-blocking failures; added a regression proving `blocking: false` yields `pass-with-review`, prints a warning check, and exits successfully.
|
|
703
|
+
- CLI smoke: built `dist`, wrote a temporary design validator/config under `/tmp/reflection-phase-4-1-smoke`, ran `node /opt/data/workspace/repos/reflection/dist/cli.js run --config /tmp/reflection-phase-4-1-smoke/reflection.config.mjs --report-dir /tmp/reflection-phase-4-1-smoke/artifacts --mode design`, then read back `report.json` and `design/tokens.log`; the report was `pass`, check `design.tokens` was blocking/pass, summary said token/source contract, and the log contained stdout and stderr details.
|
|
704
|
+
- Verification: focused design/config/report tests, `corepack pnpm typecheck`, `corepack pnpm test` (18 files / 91 tests), `corepack pnpm build`, CLI smoke, and `git diff --check` passed.
|
|
705
|
+
|
|
706
|
+
### Phase 4.2 — Family-level normalization
|
|
707
|
+
|
|
708
|
+
**Objective:** Allow validators to emit structured JSON for richer family-level checks.
|
|
709
|
+
|
|
710
|
+
**Acceptance:**
|
|
711
|
+
|
|
712
|
+
- If command emits Reflection-compatible JSON, preserve family/target metadata.
|
|
713
|
+
- If not, produce a global design check result.
|
|
714
|
+
|
|
715
|
+
**Phase 4.2 evidence — 2026-06-01:**
|
|
716
|
+
|
|
717
|
+
- RED: added a structured JSON regression proving `reflection: "design-checks-v1"` output must produce per-family checks instead of collapsing to one global `design.<command>` check.
|
|
718
|
+
- GREEN: added structured design output parsing with Zod, preserving check id, family, target, summary, details, metadata, severity, shared log artifact, command metadata, and token/source contract classification; invalid or non-JSON stdout still falls back to the global command check.
|
|
719
|
+
- Review hardening: independent review found two edge cases; fixed omitted severity so `blocking: false` commands default structured findings to review severity, and added a separate command failure check when valid structured JSON is emitted but the validator process exits non-zero.
|
|
720
|
+
- Built CLI smoke: built `dist`, ran `node /opt/data/workspace/repos/reflection/dist/cli.js run --config <temp>/reflection.config.mjs --report-dir <temp>/artifacts --mode design` in fresh temp dir `/tmp/reflection-phase-4-2-smoke-1780327323-68819`, then read back `report.json`; the report was `pass-with-review`, preserved `design.button.primary.tokens` target `primary-button` family `button`, preserved `design.card.spacing` target `marketing-card` family `card`, and `design/tokens.log` contained the structured JSON.
|
|
721
|
+
- Verification: `corepack pnpm exec vitest run tests/integration/design-command-adapter.test.ts` (9 tests), `corepack pnpm typecheck`, `corepack pnpm test` (18 files / 94 tests), `corepack pnpm build`, built CLI smoke, and `git diff --check` passed.
|
|
722
|
+
|
|
723
|
+
---
|
|
724
|
+
|
|
725
|
+
## Day 5: Storybook component visual parity
|
|
726
|
+
|
|
727
|
+
### Day 5 goal
|
|
728
|
+
|
|
729
|
+
Compare one cached expected component image against one Storybook-rendered state.
|
|
730
|
+
|
|
731
|
+
### Phase 5.1 — Storybook server and index lookup
|
|
732
|
+
|
|
733
|
+
**Objective:** Start/reuse Storybook and resolve story URLs.
|
|
734
|
+
|
|
735
|
+
**Files:**
|
|
736
|
+
- Create: `src/integrations/storybook/server.ts`
|
|
737
|
+
- Create: `src/integrations/storybook/index-json.ts`
|
|
738
|
+
- Create: `src/integrations/storybook/story-url.ts`
|
|
739
|
+
- Test: `tests/integration/storybook-index.test.ts`
|
|
740
|
+
|
|
741
|
+
**Acceptance:**
|
|
742
|
+
|
|
743
|
+
- `storyId` resolves through `/index.json`.
|
|
744
|
+
- Storybook server is started/reused like app server.
|
|
745
|
+
- Missing story produces a clear setup/config failure.
|
|
746
|
+
|
|
747
|
+
**Phase 5.1 evidence — 2026-06-01:**
|
|
748
|
+
|
|
749
|
+
- Pre-flight: verified Day 4 was committed at `dbb21a5 Add structured design command checks` and the working tree was clean before starting Day 5.
|
|
750
|
+
- RED: added `tests/integration/storybook-index.test.ts`; the focused test initially failed because `src/integrations/storybook/index-json.ts`, `server.ts`, and `story-url.ts` did not exist.
|
|
751
|
+
- GREEN: added Storybook `/index.json` loading with schema validation, story-id to `iframe.html?id=<storyId>` URL resolution, clear setup/config errors for missing/non-story entries, and `startStorybookServer` on top of the existing managed app-server lifecycle.
|
|
752
|
+
- Review hardening: independent review found path-prefixed Storybook URLs would be collapsed to the origin root. Added a RED regression for `baseUrl` ending in `/storybook/`, then changed index, iframe, and readiness URL construction to preserve base paths. Focused re-review found no remaining issues.
|
|
753
|
+
- Built module smoke: built `dist`, started a temporary HTTP Storybook fixture under `/storybook/`, called `startStorybookServer` and `resolveStoryUrl` from built JS, read back result JSON from `/tmp/reflection-phase-5-1-smoke-1780328993-83773`; the server was reused, `storyUrl` resolved to `/storybook/iframe.html?id=button--primary`, and missing story produced a setup/config error pointing at `/storybook/index.json`.
|
|
754
|
+
- Verification: focused Storybook/server-manager tests, `corepack pnpm typecheck`, `corepack pnpm test` (19 files / 99 tests), `corepack pnpm build`, built module smoke, and `git diff --check` passed.
|
|
755
|
+
|
|
756
|
+
### Phase 5.2 — Component visual case
|
|
757
|
+
|
|
758
|
+
**Objective:** Capture one deterministic component state and compare it to a cached expected image.
|
|
759
|
+
|
|
760
|
+
**Acceptance:**
|
|
761
|
+
|
|
762
|
+
- One button case produces expected/actual/diff artifacts.
|
|
763
|
+
- Result is review-only by default.
|
|
764
|
+
- Strict mode can make it blocking for selected stable cases.
|
|
765
|
+
|
|
766
|
+
**Phase 5.2 evidence — 2026-06-01:**
|
|
767
|
+
|
|
768
|
+
- RED: added `tests/integration/component-visual.test.ts`; the focused test initially failed because `src/contracts/component/component-visual-contract.ts` did not exist.
|
|
769
|
+
- GREEN: added `contracts.component` config parsing, `reflection run --mode visual` component wiring, Storybook story screenshot capture, baseline comparison, and expected/actual/diff artifact reporting.
|
|
770
|
+
- Strict behavior: review-only is the default; `strict: true`/`blocking: true` makes mismatching stable component cases blocking.
|
|
771
|
+
- Built CLI smoke: ran built `dist/cli.js` against a temporary Storybook fixture and baseline under `/tmp/reflection-phase-5-2-smoke-2btwNV`; CLI exited 0, report status was `pass`, `visual.button-primary` was `review`, and expected/actual/diff artifacts were recorded.
|
|
772
|
+
- Verification: focused component/Storybook/visual-smoke tests, `corepack pnpm typecheck`, `corepack pnpm test` (20 files / 101 tests), `corepack pnpm build`, built CLI smoke, and `git diff --check` passed. Independent review passed with no security concerns or logic errors.
|
|
773
|
+
|
|
774
|
+
### Phase 5.3 — Pseudo-state policy
|
|
775
|
+
|
|
776
|
+
**Objective:** Prefer explicit story states before browser-forced pseudo states.
|
|
777
|
+
|
|
778
|
+
**Acceptance:**
|
|
779
|
+
|
|
780
|
+
- Config supports a state note, but implementation first uses story args/decorators.
|
|
781
|
+
- Browser hover/focus can be added only with animation stabilization.
|
|
782
|
+
|
|
783
|
+
**Phase 5.3 evidence — 2026-06-01:**
|
|
784
|
+
|
|
785
|
+
- RED: added config and component visual tests for `stateNote` preservation, story-controlled state metadata, and rejection of browser-forced pseudo states without `animationStabilization`; focused tests failed for the expected missing schema/metadata behavior.
|
|
786
|
+
- GREEN: added component `stateNote` metadata, explicit `statePolicy`, and guarded `browserState` support for hover/focus with required effective animation stabilization (`disableAnimations: true` or `waitMs > 0`).
|
|
787
|
+
- Policy behavior: story args/decorators remain the default documented path (`statePolicy: story-controlled`); browser-forced hover/focus is only accepted when stabilization is declared and is reported as `browser-forced-with-stabilization`.
|
|
788
|
+
- Built CLI smoke: ran built `dist/cli.js` against a temporary Storybook hover fixture and hovered baseline under `/tmp/reflection-phase-5-3-smoke-Svz06j`; CLI exited 0, report status was `pass`, and `visual.button-primary-hover` preserved `stateNote`, `browserState`, and `statePolicy` metadata.
|
|
789
|
+
- Verification: focused config/component/Storybook tests, `corepack pnpm typecheck`, `corepack pnpm test` (20 files / 107 tests), `corepack pnpm build`, built CLI smoke, and `git diff --check` passed. Independent review initially found ineffective stabilization and missing-baseline metadata gaps; both were converted into regressions and fixed.
|
|
790
|
+
|
|
791
|
+
---
|
|
792
|
+
|
|
793
|
+
## Day 6: External spec adapter seam
|
|
794
|
+
|
|
795
|
+
### Day 6 goal
|
|
796
|
+
|
|
797
|
+
Define a clean adapter boundary for future external spec systems without making Reflection depend on them.
|
|
798
|
+
|
|
799
|
+
### Phase 6.1 — Target IR
|
|
800
|
+
|
|
801
|
+
**Objective:** Introduce a small internal representation for route/component/design targets.
|
|
802
|
+
|
|
803
|
+
**Acceptance:**
|
|
804
|
+
|
|
805
|
+
- Reflection config compiles into target IR.
|
|
806
|
+
- External adapters can later compile into the same IR.
|
|
807
|
+
- No external product name appears in core user-facing commands.
|
|
808
|
+
|
|
809
|
+
**Phase 6.1 evidence — 2026-06-01:**
|
|
810
|
+
|
|
811
|
+
- RED: added `tests/unit/target-ir.test.ts`; the focused test initially failed because `src/core/target-ir.ts` did not exist.
|
|
812
|
+
- GREEN: added `compileReflectionTargets` with a generic target IR for browser routes, route visuals, component visuals, and design commands, using neutral `reflection-config` / `adapter` source semantics.
|
|
813
|
+
- Adapter seam: target families now preserve enough normalized route/story/visual/command metadata for future optional adapters to emit the same shape without coupling core commands to an external product name. Reviewer noted falsy-but-valid threshold preservation as a hardening concern; zero-valued thresholds were added to regression coverage and preserved explicitly.
|
|
814
|
+
- Verification: `corepack pnpm exec vitest run tests/unit/target-ir.test.ts`, `corepack pnpm typecheck`, `corepack pnpm test` (21 files / 109 tests), `corepack pnpm build`, and `git diff --check` passed.
|
|
815
|
+
|
|
816
|
+
### Phase 6.2 — Adapter proof
|
|
817
|
+
|
|
818
|
+
**Objective:** Add a fixture adapter that converts a JSON route manifest into Reflection route targets.
|
|
819
|
+
|
|
820
|
+
**Acceptance:**
|
|
821
|
+
|
|
822
|
+
- Adapter is optional.
|
|
823
|
+
- Core browser runner does not know where targets came from.
|
|
824
|
+
- This validates future integration without coupling.
|
|
825
|
+
|
|
826
|
+
**Phase 6.2 evidence — 2026-06-01:**
|
|
827
|
+
|
|
828
|
+
- RED: added `tests/unit/route-manifest-adapter.test.ts`; the focused test initially failed because `src/adapters/route-manifest.ts` did not exist.
|
|
829
|
+
- GREEN: added `parseRouteManifestTargets` and `loadRouteManifestTargets`, validating a small JSON route manifest and compiling it into adapter-sourced `browser-route` Target IR. Independent review suggested stricter expectation and id validation; ambiguous expectation objects and duplicate route ids are now rejected before runner execution.
|
|
830
|
+
- Documentation: added `docs/target-ir-and-adapters.md` and linked it from the README so Target IR, adapter constraints, and the route manifest proof are documented in the docs folder.
|
|
831
|
+
- Coupling check: the adapter emits generic Target IR and no browser runner changes were needed; runner code remains unaware of route manifest source.
|
|
832
|
+
- Verification: focused route-manifest adapter tests, `corepack pnpm typecheck`, `corepack pnpm test`, `corepack pnpm build`, and `git diff --check` passed.
|
|
833
|
+
|
|
834
|
+
---
|
|
835
|
+
|
|
836
|
+
## Day 7: Product hardening and docs
|
|
837
|
+
|
|
838
|
+
### Day 7 goal
|
|
839
|
+
|
|
840
|
+
Make the project understandable and ready for continued dogfooding.
|
|
841
|
+
|
|
842
|
+
### Phase 7.1 — Documentation pass
|
|
843
|
+
|
|
844
|
+
**Files:**
|
|
845
|
+
- Create: `docs/getting-started.md`
|
|
846
|
+
- Create: `docs/configuration.md`
|
|
847
|
+
- Create: `docs/browser-contract.md`
|
|
848
|
+
- Create: `docs/visual-contract.md`
|
|
849
|
+
- Create: `docs/artifacts-and-gc.md`
|
|
850
|
+
- Create: `docs/agent-workflows.md`
|
|
851
|
+
|
|
852
|
+
**Acceptance:**
|
|
853
|
+
|
|
854
|
+
- A new repo can manually add Reflection with a minimal config.
|
|
855
|
+
- Docs clearly distinguish evidence screenshots from baselines.
|
|
856
|
+
- Docs explain that visual diffs are review-only by default.
|
|
857
|
+
|
|
858
|
+
**Phase 7.1 evidence — 2026-06-01:**
|
|
859
|
+
|
|
860
|
+
- RED: added `tests/unit/package-surface.test.ts`; the focused test initially failed because `src/index.ts` did not exist, proving the documented `import { defineReflection } from 'reflection-check'` package surface was not yet wired.
|
|
861
|
+
- GREEN: added a public package entrypoint (`src/index.ts`) plus `main`, `types`, and `exports["."]` in `package.json` so documented config files can import `defineReflection` from the package root after build.
|
|
862
|
+
- Documentation: added `getting-started`, `configuration`, `browser-contract`, `visual-contract`, `artifacts-and-gc`, and `agent-workflows` docs, and expanded the README documentation index.
|
|
863
|
+
- Acceptance smoke: verified required docs exist, README links are present, evidence screenshots are distinguished from baselines, and visual diffs are documented as review-only by default.
|
|
864
|
+
- Independent review: review caught docs/code mismatches around default config loading, config-defined run modes, component baseline promotion, browser artifact paths, manifest semantics, and CI review report roots; docs were corrected to require `--config`, document CLI-selected run modes, document current route-level baseline promotion, use the actual `browser/<route>/<viewport>` artifact layout, narrow manifest claims to report-file retention, and pass `--report-dir artifacts/reflection` for CI review.
|
|
865
|
+
- Validation: `corepack pnpm exec vitest run tests/unit/package-surface.test.ts tests/unit/config.test.ts`, `corepack pnpm typecheck`, `corepack pnpm test`, `corepack pnpm build`, and `git diff --check` passed after review fixes. Final independent re-review passed with no security concerns or logic errors.
|
|
866
|
+
|
|
867
|
+
### Phase 7.2 — Dogfood decision gate
|
|
868
|
+
|
|
869
|
+
**Objective:** Decide whether Reflection is ready for real project dogfooding.
|
|
870
|
+
|
|
871
|
+
**Acceptance checklist:**
|
|
872
|
+
|
|
873
|
+
- `reflection run` works locally against `examples/basic-react`.
|
|
874
|
+
- Browser contract catches route/layout/console failures.
|
|
875
|
+
- Reports are understandable without raw logs.
|
|
876
|
+
- Agents can parse `report.json` and identify next steps.
|
|
877
|
+
- GC cannot delete baselines or unrelated files.
|
|
878
|
+
- Visual strictness remains opt-in.
|
|
879
|
+
- No public names contain `greenhouse`.
|
|
880
|
+
|
|
881
|
+
---
|
|
882
|
+
|
|
883
|
+
## First implementation recommendation
|
|
884
|
+
|
|
885
|
+
Start Day 1 with these exact first tasks:
|
|
886
|
+
|
|
887
|
+
1. Install dependencies and verify TypeScript skeleton builds.
|
|
888
|
+
2. Move CLI command bodies out of `src/cli.ts` into command modules.
|
|
889
|
+
3. Add config schema and `defineReflection` helper.
|
|
890
|
+
4. Add report schema and artifact writer with dummy checks.
|
|
891
|
+
5. Add basic React fixture app.
|
|
892
|
+
6. Add server manager.
|
|
893
|
+
7. Add Playwright browser route runner.
|
|
894
|
+
8. Add DOM/layout/console assertions.
|
|
895
|
+
9. Run end-to-end against fixture routes.
|
|
896
|
+
10. Commit only after `pnpm typecheck`, `pnpm test`, and `pnpm build` are clean.
|
|
897
|
+
|
|
898
|
+
If tool budget or runtime setup gets tight, stop after a clean committed boundary. Do not leave half-written browser runner files without tests.
|