oh-my-codex 0.16.4 → 0.17.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/Cargo.lock +5 -5
- package/Cargo.toml +1 -1
- package/dist/catalog/__tests__/generator.test.js +2 -0
- package/dist/catalog/__tests__/generator.test.js.map +1 -1
- package/dist/cli/__tests__/doctor-warning-copy.test.js +80 -7
- package/dist/cli/__tests__/doctor-warning-copy.test.js.map +1 -1
- package/dist/cli/__tests__/index.test.js +17 -11
- package/dist/cli/__tests__/index.test.js.map +1 -1
- package/dist/cli/__tests__/mcp-serve.test.js +4 -0
- package/dist/cli/__tests__/mcp-serve.test.js.map +1 -1
- package/dist/cli/__tests__/setup-hooks-shared-ownership.test.js +8 -3
- package/dist/cli/__tests__/setup-hooks-shared-ownership.test.js.map +1 -1
- package/dist/cli/__tests__/setup-install-mode.test.js +27 -1
- package/dist/cli/__tests__/setup-install-mode.test.js.map +1 -1
- package/dist/cli/__tests__/ultragoal.test.js +22 -0
- package/dist/cli/__tests__/ultragoal.test.js.map +1 -1
- package/dist/cli/doctor.d.ts.map +1 -1
- package/dist/cli/doctor.js +66 -10
- package/dist/cli/doctor.js.map +1 -1
- package/dist/cli/index.d.ts +8 -2
- package/dist/cli/index.d.ts.map +1 -1
- package/dist/cli/index.js +17 -7
- package/dist/cli/index.js.map +1 -1
- package/dist/cli/mcp-serve.d.ts.map +1 -1
- package/dist/cli/mcp-serve.js +4 -0
- package/dist/cli/mcp-serve.js.map +1 -1
- package/dist/cli/plugin-marketplace.d.ts +20 -0
- package/dist/cli/plugin-marketplace.d.ts.map +1 -1
- package/dist/cli/plugin-marketplace.js +115 -1
- package/dist/cli/plugin-marketplace.js.map +1 -1
- package/dist/cli/setup.d.ts.map +1 -1
- package/dist/cli/setup.js +29 -10
- package/dist/cli/setup.js.map +1 -1
- package/dist/cli/ultragoal.d.ts.map +1 -1
- package/dist/cli/ultragoal.js +7 -1
- package/dist/cli/ultragoal.js.map +1 -1
- package/dist/config/__tests__/codex-hooks.test.js +136 -9
- package/dist/config/__tests__/codex-hooks.test.js.map +1 -1
- package/dist/config/__tests__/generator-idempotent.test.js +15 -0
- package/dist/config/__tests__/generator-idempotent.test.js.map +1 -1
- package/dist/config/codex-hooks.d.ts +13 -14
- package/dist/config/codex-hooks.d.ts.map +1 -1
- package/dist/config/codex-hooks.js +85 -7
- package/dist/config/codex-hooks.js.map +1 -1
- package/dist/config/generator.d.ts +4 -1
- package/dist/config/generator.d.ts.map +1 -1
- package/dist/config/generator.js +15 -9
- package/dist/config/generator.js.map +1 -1
- package/dist/config/omx-first-party-mcp.d.ts.map +1 -1
- package/dist/config/omx-first-party-mcp.js +7 -0
- package/dist/config/omx-first-party-mcp.js.map +1 -1
- package/dist/hooks/__tests__/design-skill.test.d.ts +2 -0
- package/dist/hooks/__tests__/design-skill.test.d.ts.map +1 -0
- package/dist/hooks/__tests__/design-skill.test.js +55 -0
- package/dist/hooks/__tests__/design-skill.test.js.map +1 -0
- package/dist/hooks/__tests__/notify-hook-tmux-heal.test.js +265 -0
- package/dist/hooks/__tests__/notify-hook-tmux-heal.test.js.map +1 -1
- package/dist/hooks/__tests__/skill-catalog-hygiene.test.js +1 -1
- package/dist/hooks/__tests__/skill-catalog-hygiene.test.js.map +1 -1
- package/dist/hooks/__tests__/skill-guidance-contract.test.js +41 -0
- package/dist/hooks/__tests__/skill-guidance-contract.test.js.map +1 -1
- package/dist/hooks/keyword-detector.d.ts.map +1 -1
- package/dist/hooks/keyword-detector.js +5 -1
- package/dist/hooks/keyword-detector.js.map +1 -1
- package/dist/hooks/keyword-registry.d.ts.map +1 -1
- package/dist/hooks/keyword-registry.js +2 -0
- package/dist/hooks/keyword-registry.js.map +1 -1
- package/dist/hooks/prompt-guidance-contract.d.ts.map +1 -1
- package/dist/hooks/prompt-guidance-contract.js +47 -2
- package/dist/hooks/prompt-guidance-contract.js.map +1 -1
- package/dist/mcp/__tests__/bootstrap.test.js +3 -0
- package/dist/mcp/__tests__/bootstrap.test.js.map +1 -1
- package/dist/mcp/__tests__/hermes-bridge.test.d.ts +2 -0
- package/dist/mcp/__tests__/hermes-bridge.test.d.ts.map +1 -0
- package/dist/mcp/__tests__/hermes-bridge.test.js +374 -0
- package/dist/mcp/__tests__/hermes-bridge.test.js.map +1 -0
- package/dist/mcp/__tests__/state-paths.test.js +96 -13
- package/dist/mcp/__tests__/state-paths.test.js.map +1 -1
- package/dist/mcp/bootstrap.d.ts +1 -1
- package/dist/mcp/bootstrap.d.ts.map +1 -1
- package/dist/mcp/bootstrap.js +2 -0
- package/dist/mcp/bootstrap.js.map +1 -1
- package/dist/mcp/hermes-bridge.d.ts +81 -0
- package/dist/mcp/hermes-bridge.d.ts.map +1 -0
- package/dist/mcp/hermes-bridge.js +400 -0
- package/dist/mcp/hermes-bridge.js.map +1 -0
- package/dist/mcp/hermes-server.d.ts +269 -0
- package/dist/mcp/hermes-server.d.ts.map +1 -0
- package/dist/mcp/hermes-server.js +121 -0
- package/dist/mcp/hermes-server.js.map +1 -0
- package/dist/mcp/state-paths.d.ts.map +1 -1
- package/dist/mcp/state-paths.js +41 -9
- package/dist/mcp/state-paths.js.map +1 -1
- package/dist/modes/__tests__/base-tmux-pane.test.js +31 -1
- package/dist/modes/__tests__/base-tmux-pane.test.js.map +1 -1
- package/dist/scripts/__tests__/codex-native-hook.test.js +187 -2
- package/dist/scripts/__tests__/codex-native-hook.test.js.map +1 -1
- package/dist/scripts/codex-native-hook.d.ts +1 -0
- package/dist/scripts/codex-native-hook.d.ts.map +1 -1
- package/dist/scripts/codex-native-hook.js +44 -17
- package/dist/scripts/codex-native-hook.js.map +1 -1
- package/dist/scripts/notify-hook/tmux-injection.d.ts.map +1 -1
- package/dist/scripts/notify-hook/tmux-injection.js +91 -2
- package/dist/scripts/notify-hook/tmux-injection.js.map +1 -1
- package/dist/state/mode-state-context.d.ts +2 -0
- package/dist/state/mode-state-context.d.ts.map +1 -1
- package/dist/state/mode-state-context.js +21 -0
- package/dist/state/mode-state-context.js.map +1 -1
- package/dist/ultragoal/__tests__/artifacts.test.js +121 -0
- package/dist/ultragoal/__tests__/artifacts.test.js.map +1 -1
- package/dist/ultragoal/artifacts.d.ts +9 -1
- package/dist/ultragoal/artifacts.d.ts.map +1 -1
- package/dist/ultragoal/artifacts.js +105 -3
- package/dist/ultragoal/artifacts.js.map +1 -1
- package/dist/utils/__tests__/paths.test.js +31 -1
- package/dist/utils/__tests__/paths.test.js.map +1 -1
- package/dist/utils/paths.d.ts +6 -0
- package/dist/utils/paths.d.ts.map +1 -1
- package/dist/utils/paths.js +18 -0
- package/dist/utils/paths.js.map +1 -1
- package/dist/wiki/lifecycle.js +3 -3
- package/dist/wiki/lifecycle.js.map +1 -1
- package/package.json +1 -1
- package/plugins/oh-my-codex/.codex-plugin/plugin.json +1 -1
- package/plugins/oh-my-codex/.mcp.json +8 -0
- package/plugins/oh-my-codex/skills/design/SKILL.md +180 -0
- package/plugins/oh-my-codex/skills/skill/SKILL.md +2 -1
- package/plugins/oh-my-codex/skills/ultraqa/SKILL.md +161 -47
- package/plugins/oh-my-codex/skills/visual-ralph/SKILL.md +2 -2
- package/skills/design/SKILL.md +180 -0
- package/skills/frontend-ui-ux/SKILL.md +6 -2
- package/skills/skill/SKILL.md +2 -1
- package/skills/ultraqa/SKILL.md +161 -47
- package/skills/visual-ralph/SKILL.md +2 -2
- package/src/scripts/__tests__/codex-native-hook.test.ts +206 -1
- package/src/scripts/codex-native-hook.ts +45 -18
- package/src/scripts/notify-hook/tmux-injection.ts +110 -3
- package/templates/catalog-manifest.json +9 -2
|
@@ -0,0 +1,180 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: design
|
|
3
|
+
description: Canonical repo-local DESIGN.md workflow for product, UI/UX, and frontend decision source of truth
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Design Skill
|
|
7
|
+
|
|
8
|
+
Use `$design` when product, UI/UX, frontend, or design-system decisions need a durable source of truth in the repository. This skill discovers existing design context, interviews for missing product/design information, and creates or refreshes repo-local `DESIGN.md` so future UI/UX/frontend work is grounded instead of improvised.
|
|
9
|
+
|
|
10
|
+
## Purpose
|
|
11
|
+
|
|
12
|
+
Make repo-local `DESIGN.md` source of truth and canonical design contract for the current repository:
|
|
13
|
+
|
|
14
|
+
`existing repo evidence -> missing-context interview -> create/refresh DESIGN.md -> use DESIGN.md for UI/UX/frontend decisions`.
|
|
15
|
+
|
|
16
|
+
The output is not a pixel-matching loop and not a one-off visual critique. It is the maintained design brief/checklist that implementation, review, and future visual work should cite.
|
|
17
|
+
|
|
18
|
+
## Use when
|
|
19
|
+
|
|
20
|
+
- The user asks for design direction, UX guidance, frontend planning, or design-system alignment.
|
|
21
|
+
- A repo needs a design brief before UI/frontend implementation begins.
|
|
22
|
+
- Existing UI/components/assets/screenshots need to be summarized into a reusable design source of truth.
|
|
23
|
+
- UI/UX/frontend decisions are ambiguous and should be resolved through product context, constraints, and documented principles.
|
|
24
|
+
- A feature needs `DESIGN.md` created or refreshed before `$ralph`, a designer lane, or implementation work proceeds.
|
|
25
|
+
|
|
26
|
+
## Do not use when
|
|
27
|
+
|
|
28
|
+
- The user provides or requests a visual reference/image/live URL and wants measured implementation until screenshots match. Use `$visual-ralph` for that visual-reference implementation loop.
|
|
29
|
+
- The task is pure backend/API/infrastructure work with no user-facing design consequence.
|
|
30
|
+
- The user only asks to compare screenshots or score visual fidelity. Use `$visual-ralph` and its built-in visual verdict flow.
|
|
31
|
+
|
|
32
|
+
## Relationship to `$visual-ralph`
|
|
33
|
+
|
|
34
|
+
`$design` owns the durable repo design source of truth: product goals, users, IA, visual language, components, accessibility, constraints, and open questions in `DESIGN.md`.
|
|
35
|
+
|
|
36
|
+
`$visual-ralph` owns implementation against an approved generated/static/live-URL visual reference, with screenshot capture, Visual Ralph verdict scoring, and pixel-diff evidence. `$visual-ralph` may read `DESIGN.md`, and it may leave design-system artifacts behind, but it does not replace the `DESIGN.md` discovery/interview/refresh workflow.
|
|
37
|
+
|
|
38
|
+
If both are needed, run `$design` first to establish the design contract, then run `$visual-ralph` only after the visual reference/baseline is approved.
|
|
39
|
+
|
|
40
|
+
## Workflow
|
|
41
|
+
|
|
42
|
+
### 1. Discover local design evidence
|
|
43
|
+
|
|
44
|
+
Inspect the repository before writing guidance. Look for:
|
|
45
|
+
|
|
46
|
+
- `DESIGN.md`, `docs/design*`, `docs/ux*`, `docs/frontend*`, `README.md`, product specs, PRDs, and issue notes.
|
|
47
|
+
- Existing UI source: routes, pages, layouts, components, stories, examples, demos, theme files, CSS variables, Tailwind/theme config, tokens, icons, and assets.
|
|
48
|
+
- Screenshots, mockups, brand files, logos, Figma/export notes, Storybook snapshots, Playwright screenshots, visual-regression baselines, or `.omx/artifacts/visual-ralph/*` references.
|
|
49
|
+
- Accessibility, responsive, i18n, content, and platform constraints already encoded in code or docs.
|
|
50
|
+
|
|
51
|
+
Record evidence with file paths. Distinguish observed facts from design inferences.
|
|
52
|
+
|
|
53
|
+
### 2. Interview only for missing context
|
|
54
|
+
|
|
55
|
+
Ask concise questions only when repo evidence cannot answer design-critical context. Prefer one focused round that closes the biggest gaps, such as:
|
|
56
|
+
|
|
57
|
+
- target users/personas and jobs to be done,
|
|
58
|
+
- product/business goals and non-goals,
|
|
59
|
+
- brand personality or forbidden aesthetics,
|
|
60
|
+
- primary flows and information architecture,
|
|
61
|
+
- accessibility level, device/browser support, and implementation constraints,
|
|
62
|
+
- existing design assets or references the repo does not contain.
|
|
63
|
+
|
|
64
|
+
If the user wants autonomous progress or cannot answer, create `DESIGN.md` with explicit assumptions and open questions instead of blocking.
|
|
65
|
+
|
|
66
|
+
### 3. Create or refresh `DESIGN.md`
|
|
67
|
+
|
|
68
|
+
Use the structure below. Preserve useful existing content, remove contradictions, and mark unknowns as open questions. Keep it actionable for implementers and reviewers.
|
|
69
|
+
|
|
70
|
+
#### Required `DESIGN.md` structure/checklist
|
|
71
|
+
|
|
72
|
+
```markdown
|
|
73
|
+
# Design
|
|
74
|
+
|
|
75
|
+
## Source of truth
|
|
76
|
+
- Status: Draft | Active | Needs refresh
|
|
77
|
+
- Last refreshed: YYYY-MM-DD
|
|
78
|
+
- Primary product surfaces:
|
|
79
|
+
- Evidence reviewed:
|
|
80
|
+
|
|
81
|
+
## Brand
|
|
82
|
+
- Personality:
|
|
83
|
+
- Trust signals:
|
|
84
|
+
- Avoid:
|
|
85
|
+
|
|
86
|
+
## Product goals
|
|
87
|
+
- Goals:
|
|
88
|
+
- Non-goals:
|
|
89
|
+
- Success signals:
|
|
90
|
+
|
|
91
|
+
## Personas and jobs
|
|
92
|
+
- Primary personas:
|
|
93
|
+
- User jobs:
|
|
94
|
+
- Key contexts of use:
|
|
95
|
+
|
|
96
|
+
## Information architecture
|
|
97
|
+
- Primary navigation:
|
|
98
|
+
- Core routes/screens:
|
|
99
|
+
- Content hierarchy:
|
|
100
|
+
|
|
101
|
+
## Design principles
|
|
102
|
+
- Principle 1:
|
|
103
|
+
- Principle 2:
|
|
104
|
+
- Tradeoffs:
|
|
105
|
+
|
|
106
|
+
## Visual language
|
|
107
|
+
- Color:
|
|
108
|
+
- Typography:
|
|
109
|
+
- Spacing/layout rhythm:
|
|
110
|
+
- Shape/radius/elevation:
|
|
111
|
+
- Motion:
|
|
112
|
+
- Imagery/iconography:
|
|
113
|
+
|
|
114
|
+
## Components
|
|
115
|
+
- Existing components to reuse:
|
|
116
|
+
- New/changed components:
|
|
117
|
+
- Variants and states:
|
|
118
|
+
- Token/component ownership:
|
|
119
|
+
|
|
120
|
+
## Accessibility
|
|
121
|
+
- Target standard:
|
|
122
|
+
- Keyboard/focus behavior:
|
|
123
|
+
- Contrast/readability:
|
|
124
|
+
- Screen-reader semantics:
|
|
125
|
+
- Reduced motion and sensory considerations:
|
|
126
|
+
|
|
127
|
+
## Responsive behavior
|
|
128
|
+
- Supported breakpoints/devices:
|
|
129
|
+
- Layout adaptations:
|
|
130
|
+
- Touch/hover differences:
|
|
131
|
+
|
|
132
|
+
## Interaction states
|
|
133
|
+
- Loading:
|
|
134
|
+
- Empty:
|
|
135
|
+
- Error:
|
|
136
|
+
- Success:
|
|
137
|
+
- Disabled:
|
|
138
|
+
- Offline/slow network, if applicable:
|
|
139
|
+
|
|
140
|
+
## Content voice
|
|
141
|
+
- Tone:
|
|
142
|
+
- Terminology:
|
|
143
|
+
- Microcopy rules:
|
|
144
|
+
|
|
145
|
+
## Implementation constraints
|
|
146
|
+
- Framework/styling system:
|
|
147
|
+
- Design-token constraints:
|
|
148
|
+
- Performance constraints:
|
|
149
|
+
- Compatibility constraints:
|
|
150
|
+
- Test/screenshot expectations:
|
|
151
|
+
|
|
152
|
+
## Open questions
|
|
153
|
+
- [ ] Question / owner / impact
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
### 4. Use `DESIGN.md` as the decision contract
|
|
157
|
+
|
|
158
|
+
For UI/UX/frontend work after the refresh:
|
|
159
|
+
|
|
160
|
+
- Cite the relevant `DESIGN.md` sections before making design choices.
|
|
161
|
+
- Prefer existing components, tokens, and documented constraints.
|
|
162
|
+
- If implementation reveals a design contradiction, update `DESIGN.md` or add an open question before proceeding.
|
|
163
|
+
- Do not introduce a new design-system layer when existing repo-native patterns can be extended.
|
|
164
|
+
|
|
165
|
+
### 5. Handoff to implementation or Visual Ralph when appropriate
|
|
166
|
+
|
|
167
|
+
- For normal frontend implementation, hand off with the relevant `DESIGN.md` sections, repo evidence, and acceptance criteria.
|
|
168
|
+
- For visual-reference/image/live-URL matching, hand off to `$visual-ralph` with the approved reference/baseline and note that `DESIGN.md` is supporting context, not the visual verdict target.
|
|
169
|
+
|
|
170
|
+
## Completion checklist
|
|
171
|
+
|
|
172
|
+
Do not declare the design workflow complete until:
|
|
173
|
+
|
|
174
|
+
- Existing design docs/assets/components/screenshots have been inspected or explicitly noted as absent.
|
|
175
|
+
- Missing product/design context has been answered, assumed, or listed in `DESIGN.md` open questions.
|
|
176
|
+
- `DESIGN.md` exists at the repo root and contains all required checklist sections.
|
|
177
|
+
- UI/UX/frontend recommendations cite `DESIGN.md` rather than relying on unstated preferences.
|
|
178
|
+
- Any `$visual-ralph` handoff is clearly separated as visual implementation matching, not DESIGN.md governance.
|
|
179
|
+
|
|
180
|
+
Task: {{ARGUMENTS}}
|
|
@@ -1,12 +1,16 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: frontend-ui-ux
|
|
3
|
-
description: Deprecated compatibility shim for
|
|
3
|
+
description: Deprecated compatibility shim for frontend UI/UX work; use $design or $visual-ralph
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Frontend UI/UX compatibility shim
|
|
7
7
|
|
|
8
8
|
Hard-deprecated. Do not invoke or route this skill for new work.
|
|
9
9
|
|
|
10
|
-
Use
|
|
10
|
+
Use `$design` when the task needs product/design context, UX guidance, frontend planning, design-system alignment, or a repo-local `DESIGN.md` source of truth.
|
|
11
|
+
|
|
12
|
+
Use `$visual-ralph` when the task needs implementation against an approved generated/static/live-URL visual reference with screenshot capture, Visual Ralph verdict scoring, and pixel-diff evidence.
|
|
13
|
+
|
|
14
|
+
This file exists only to preserve the public/catalog-visible `frontend-ui-ux` compatibility contract while canonical design guidance is handled by `$design` and measured visual implementation is handled by `$visual-ralph`.
|
|
11
15
|
|
|
12
16
|
Task: {{ARGUMENTS}}
|
package/skills/skill/SKILL.md
CHANGED
package/skills/ultraqa/SKILL.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: ultraqa
|
|
3
|
-
description: QA
|
|
3
|
+
description: Adversarial dynamic e2e QA workflow - generate hostile scenarios, test, verify, fix, report, and clean up
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# UltraQA Skill
|
|
@@ -10,84 +10,184 @@ description: QA cycling workflow - test, verify, fix, repeat until goal met
|
|
|
10
10
|
- Use outcome-first framing with concise, evidence-dense progress and completion reporting.
|
|
11
11
|
- Treat newer user updates as local overrides for the active workflow branch while preserving earlier non-conflicting constraints.
|
|
12
12
|
- If the user says `continue`, advance the current verified next step instead of restarting discovery.
|
|
13
|
+
- UltraQA is not satisfied by a shallow build/lint/typecheck/test checklist. It must exercise the requested behavior through adversarial dynamic e2e scenarios whenever the target can be run, simulated, or harnessed safely.
|
|
13
14
|
|
|
14
|
-
[ULTRAQA ACTIVATED -
|
|
15
|
+
[ULTRAQA ACTIVATED - ADVERSARIAL DYNAMIC E2E QA CYCLING]
|
|
15
16
|
|
|
16
17
|
## Overview
|
|
17
18
|
|
|
19
|
+
UltraQA finds real behavior failures by combining normal verification commands with generated end-to-end scenarios, hostile user modeling, temporary harnesses when useful, and a structured evidence report. The workflow repeats test → diagnose → fix → retest until the goal is met, a bounded stop condition is reached, or a safety boundary blocks further execution.
|
|
20
|
+
|
|
18
21
|
## Goal Parsing
|
|
19
22
|
|
|
20
23
|
Parse the goal from arguments. Supported formats:
|
|
21
24
|
|
|
22
25
|
| Invocation | Goal Type | What to Check |
|
|
23
26
|
|------------|-----------|---------------|
|
|
24
|
-
| `/ultraqa --tests` | tests |
|
|
25
|
-
| `/ultraqa --build` | build | Build succeeds
|
|
26
|
-
| `/ultraqa --lint` | lint |
|
|
27
|
-
| `/ultraqa --typecheck` | typecheck |
|
|
28
|
-
| `/ultraqa --custom "pattern"` | custom | Custom success pattern
|
|
27
|
+
| `/ultraqa --tests` | tests | Existing tests plus adversarial dynamic e2e scenarios for the changed behavior |
|
|
28
|
+
| `/ultraqa --build` | build | Build succeeds and generated smoke/e2e probes still run against the built artifact when applicable |
|
|
29
|
+
| `/ultraqa --lint` | lint | Lint passes and no generated harness/test artifact violates project hygiene |
|
|
30
|
+
| `/ultraqa --typecheck` | typecheck | Typecheck passes and generated typed harnesses compile when applicable |
|
|
31
|
+
| `/ultraqa --custom "pattern"` | custom | Custom success pattern is verified against behavior, not trusted as misleading success output |
|
|
32
|
+
| `/ultraqa --interactive` | interactive | CLI/service behavior is tested with generated hostile and edge-case interactions |
|
|
33
|
+
|
|
34
|
+
If no structured goal is provided, interpret the argument as a custom behavior goal and derive a runnable e2e strategy from repository context.
|
|
35
|
+
|
|
36
|
+
## Required Scenario Matrix
|
|
37
|
+
|
|
38
|
+
Before declaring success, create and maintain a scenario matrix. Each row must include: scenario id, intent, user/attacker model, setup, command or harness, expected signal, actual result, fixes applied, evidence, and cleanup status.
|
|
29
39
|
|
|
30
|
-
|
|
40
|
+
The matrix must include normal-path coverage plus adversarial dynamic e2e scenarios selected from the current goal and codebase. Unless clearly irrelevant or impossible, include these hostile and edge-case classes:
|
|
41
|
+
|
|
42
|
+
1. **Malformed input**: invalid JSON, missing fields, invalid flags, oversized strings, unusual Unicode, path traversal-like values, and corrupted state files.
|
|
43
|
+
2. **Repeated interruptions**: repeated `continue`, stop/cancel/abort wording, interrupted command output, and retries after partial progress.
|
|
44
|
+
3. **Prompt injection attempts**: user text that tries to override instructions, exfiltrate secrets, skip verification, delete state, or claim false success.
|
|
45
|
+
4. **Cancel/resume behavior**: active state cleanup, resume detection, stale in-progress state, and cancellation followed by a fresh run.
|
|
46
|
+
5. **Stale state**: old `.omx/state` files, mismatched sessions, missing timestamps, and contradictory phase metadata.
|
|
47
|
+
6. **Dirty worktree**: pre-existing modifications, untracked generated files, and verification that UltraQA does not hide or overwrite unrelated work.
|
|
48
|
+
7. **Hung or long-running commands**: bounded timeout handling, killed child processes, and recovery notes.
|
|
49
|
+
8. **Flaky tests**: rerun strategy, failure clustering, quarantine evidence, and avoiding false green from a single lucky pass.
|
|
50
|
+
9. **Misleading success output**: output containing success phrases with non-zero exits, hidden failures, skipped tests, or partial command logs.
|
|
51
|
+
|
|
52
|
+
## Dynamic E2E and Temporary Harness Rules
|
|
53
|
+
|
|
54
|
+
- Generate temporary tests, scripts, fixtures, or harnesses when they materially improve behavioral confidence and no existing e2e surface covers the scenario.
|
|
55
|
+
- Prefer project-native test tools and small throwaway harnesses under a temporary directory or clearly named test fixture.
|
|
56
|
+
- Record every generated artifact in the scenario matrix, including whether it was committed intentionally or removed during cleanup.
|
|
57
|
+
- Use bounded runtimes and explicit timeouts for commands that can hang.
|
|
58
|
+
- Validate exit codes and output semantics; do not trust success-looking text alone.
|
|
59
|
+
- Do not delete, rewrite, or mask unrelated user work. Capture dirty-worktree evidence before and after generated harness work.
|
|
31
60
|
|
|
32
61
|
## Cycle Workflow
|
|
33
62
|
|
|
34
63
|
### Cycle N (Max 5)
|
|
35
64
|
|
|
36
|
-
1. **
|
|
37
|
-
-
|
|
38
|
-
-
|
|
39
|
-
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
- `--
|
|
65
|
+
1. **PLAN ADVERSARIAL QA**
|
|
66
|
+
- Restate the goal, success criteria, safety bounds, and stop condition.
|
|
67
|
+
- Inspect repository context enough to identify runnable surfaces, test commands, state files, and cleanup paths.
|
|
68
|
+
- Build or update the required scenario matrix before running commands.
|
|
69
|
+
|
|
70
|
+
2. **RUN BASELINE VERIFICATION**
|
|
71
|
+
- `--tests`: Run the project's test command.
|
|
72
|
+
- `--build`: Run the project's build command.
|
|
73
|
+
- `--lint`: Run the project's lint command.
|
|
74
|
+
- `--typecheck`: Run the project's type check command.
|
|
75
|
+
- `--custom`: Run the appropriate command and check the pattern plus exit status and failure markers.
|
|
76
|
+
- `--interactive`: Use qa-tester or an equivalent CLI/service harness:
|
|
43
77
|
```
|
|
44
78
|
Use `/prompts:qa-tester` with:
|
|
45
79
|
Goal: [describe what to verify]
|
|
46
80
|
Service: [how to start]
|
|
47
|
-
Test cases: [
|
|
81
|
+
Test cases: [normal, hostile, malformed, interruption, resume, stale-state, dirty-worktree, hung-command, flaky, and misleading-output scenarios]
|
|
48
82
|
```
|
|
49
83
|
|
|
50
|
-
|
|
51
|
-
-
|
|
52
|
-
-
|
|
84
|
+
3. **RUN ADVERSARIAL DYNAMIC E2E SCENARIOS**
|
|
85
|
+
- Execute the scenario matrix using existing e2e tests, generated temporary tests, or generated harnesses.
|
|
86
|
+
- Model malicious/hostile user behavior explicitly, including prompt injection and attempts to bypass safety or verification.
|
|
87
|
+
- Exercise malformed input, repeated interruptions, cancel/resume, stale state, dirty worktree handling, hung commands, flaky tests, and misleading success output when relevant.
|
|
88
|
+
- Capture commands, exit codes, important output excerpts, artifacts, and cleanup status.
|
|
89
|
+
|
|
90
|
+
4. **CHECK RESULT**
|
|
91
|
+
- **YES** only if baseline verification and adversarial e2e scenarios passed, generated artifacts are cleaned up or intentionally tracked, and the report has complete evidence.
|
|
92
|
+
- **NO** if any scenario failed, was skipped without justification, left debris, relied on misleading output, or lacked evidence. Continue to step 5.
|
|
53
93
|
|
|
54
|
-
|
|
94
|
+
5. **ARCHITECT DIAGNOSIS**
|
|
55
95
|
```
|
|
56
96
|
Use `/prompts:architect` with:
|
|
57
|
-
Goal: [goal type]
|
|
58
|
-
|
|
59
|
-
|
|
97
|
+
Goal: [goal type and behavior]
|
|
98
|
+
Scenario matrix: [rows, commands, failures, evidence]
|
|
99
|
+
Output: [test/build/e2e/harness output]
|
|
100
|
+
Provide root cause, safety implications, and specific fix recommendations.
|
|
60
101
|
```
|
|
61
102
|
|
|
62
|
-
|
|
103
|
+
6. **FIX ISSUES**
|
|
63
104
|
```
|
|
64
105
|
Use `/prompts:executor` with:
|
|
65
106
|
Issue: [architect diagnosis]
|
|
66
107
|
Files: [affected files]
|
|
108
|
+
Constraints: preserve unrelated dirty work, clean temporary harnesses, keep safety bounds
|
|
67
109
|
Apply the fix precisely as recommended.
|
|
68
110
|
```
|
|
69
111
|
|
|
70
|
-
|
|
112
|
+
7. **CLEAN UP AND ROLLBACK**
|
|
113
|
+
- Remove temporary harnesses, fixtures, logs, spawned processes, and state files unless they are intentional deliverables.
|
|
114
|
+
- Roll back failed experimental edits that are not part of the final fix.
|
|
115
|
+
- Re-check the worktree and record remaining intentional changes or residual debris.
|
|
116
|
+
|
|
117
|
+
8. **REPEAT**
|
|
118
|
+
- Go back to step 1 with the updated scenario matrix and failure history.
|
|
119
|
+
|
|
120
|
+
## Safety Bounds
|
|
121
|
+
|
|
122
|
+
UltraQA must stay inside these safety bounds:
|
|
123
|
+
|
|
124
|
+
- No destructive commands such as force resets, broad deletes, secret exfiltration, credential dumping, production writes, or unbounded process spawning.
|
|
125
|
+
- No reading or printing secrets beyond the minimum metadata needed to verify absence of leakage.
|
|
126
|
+
- No network or external-production side effects unless the user explicitly authorized them.
|
|
127
|
+
- No unbounded waits: use timeouts, retries with caps, and clear hung-command diagnostics.
|
|
128
|
+
- No hiding unrelated dirty work or generated debris.
|
|
129
|
+
- If a required scenario would violate these bounds, mark it blocked in the report with the safe substitute used.
|
|
71
130
|
|
|
72
131
|
## Exit Conditions
|
|
73
132
|
|
|
74
133
|
| Condition | Action |
|
|
75
134
|
|-----------|--------|
|
|
76
|
-
| **Goal Met** | Exit with success:
|
|
77
|
-
| **Cycle 5 Reached** | Exit with diagnosis:
|
|
78
|
-
| **Same Failure 3x** | Exit early:
|
|
79
|
-
| **
|
|
135
|
+
| **Goal Met** | Exit with success: `ULTRAQA COMPLETE: Goal met after N cycles` plus the structured report |
|
|
136
|
+
| **Cycle 5 Reached** | Exit with diagnosis: `ULTRAQA STOPPED: Max cycles` plus failures, fixes attempted, residual risks, and evidence |
|
|
137
|
+
| **Same Failure 3x** | Exit early: `ULTRAQA STOPPED: Same failure detected 3 times` plus root cause, safety notes, and next owner |
|
|
138
|
+
| **Safety Boundary** | Exit: `ULTRAQA BLOCKED: [destructive/credentialed/external-production/unbounded action]` plus safe substitute evidence |
|
|
139
|
+
| **Environment Error** | Exit: `ULTRAQA ERROR: [tmux/port/dependency/hung command issue]` plus cleanup status |
|
|
140
|
+
|
|
141
|
+
## Structured Report
|
|
142
|
+
|
|
143
|
+
Every terminal UltraQA result must include this report shape:
|
|
144
|
+
|
|
145
|
+
```markdown
|
|
146
|
+
# UltraQA Report
|
|
147
|
+
|
|
148
|
+
## Goal and success criteria
|
|
149
|
+
- Goal:
|
|
150
|
+
- Stop condition:
|
|
151
|
+
- Safety bounds applied:
|
|
152
|
+
|
|
153
|
+
## Scenario matrix
|
|
154
|
+
| ID | User/attacker model | Scenario | Command/harness | Expected signal | Actual result | Status | Evidence | Cleanup |
|
|
155
|
+
|----|---------------------|----------|-----------------|-----------------|---------------|--------|----------|---------|
|
|
156
|
+
|
|
157
|
+
## Commands run
|
|
158
|
+
- `[exit code] command` — purpose, duration/timeout, key output evidence
|
|
159
|
+
|
|
160
|
+
## Failures found
|
|
161
|
+
- Scenario ID, failure signal, root cause, user impact, safety impact
|
|
162
|
+
|
|
163
|
+
## Fixes applied
|
|
164
|
+
- Files changed, rationale, linked failing scenario(s), regression evidence
|
|
165
|
+
|
|
166
|
+
## Cleanup and rollback
|
|
167
|
+
- Generated artifacts removed or intentionally kept
|
|
168
|
+
- State/process cleanup performed
|
|
169
|
+
- Worktree status before/after
|
|
170
|
+
|
|
171
|
+
## Residual risks
|
|
172
|
+
- Untested or blocked scenarios with reasons and safe substitutes
|
|
173
|
+
|
|
174
|
+
## Evidence
|
|
175
|
+
- Test output, e2e logs, harness output, screenshots/transcripts when relevant, and rerun/flake evidence
|
|
176
|
+
```
|
|
80
177
|
|
|
81
178
|
## Observability
|
|
82
179
|
|
|
83
180
|
Output progress each cycle:
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
[ULTRAQA Cycle 1/5]
|
|
87
|
-
[ULTRAQA Cycle 1/5]
|
|
88
|
-
[ULTRAQA Cycle 1/5]
|
|
89
|
-
[ULTRAQA Cycle
|
|
90
|
-
[ULTRAQA Cycle
|
|
181
|
+
|
|
182
|
+
```text
|
|
183
|
+
[ULTRAQA Cycle 1/5] Planning adversarial scenario matrix...
|
|
184
|
+
[ULTRAQA Cycle 1/5] Running baseline tests...
|
|
185
|
+
[ULTRAQA Cycle 1/5] Running ADV-E2E-003 prompt-injection harness...
|
|
186
|
+
[ULTRAQA Cycle 1/5] FAILED - stale state resume accepted misleading success output
|
|
187
|
+
[ULTRAQA Cycle 1/5] Architect diagnosing scenario ADV-E2E-003...
|
|
188
|
+
[ULTRAQA Cycle 1/5] Fixing: src/hooks/... - validate exit code before success phrase
|
|
189
|
+
[ULTRAQA Cycle 1/5] Cleaning temporary harnesses and state...
|
|
190
|
+
[ULTRAQA Cycle 2/5] PASSED - baseline + 9 adversarial scenarios pass
|
|
91
191
|
[ULTRAQA COMPLETE] Goal met after 2 cycles
|
|
92
192
|
```
|
|
93
193
|
|
|
@@ -96,12 +196,16 @@ Output progress each cycle:
|
|
|
96
196
|
Use the CLI-first state surface (`omx state ... --json`) for UltraQA lifecycle state. If explicit MCP compatibility tools are already available, equivalent `omx_state` calls are optional compatibility, not the default.
|
|
97
197
|
|
|
98
198
|
- **On start**:
|
|
99
|
-
`omx state write --input '{"mode":"ultraqa","active":true,"current_phase":"
|
|
199
|
+
`omx state write --input '{"mode":"ultraqa","active":true,"current_phase":"planning","iteration":1,"started_at":"<now>","scenario_matrix":[]}' --json`
|
|
100
200
|
- **On each cycle**:
|
|
101
|
-
`omx state write --input '{"mode":"ultraqa","current_phase":"qa","iteration":<cycle>}' --json`
|
|
201
|
+
`omx state write --input '{"mode":"ultraqa","current_phase":"qa","iteration":<cycle>,"scenario_matrix":"<updated matrix path or summary>"}' --json`
|
|
202
|
+
- **On adversarial e2e transition**:
|
|
203
|
+
`omx state write --input '{"mode":"ultraqa","current_phase":"adversarial-e2e"}' --json`
|
|
102
204
|
- **On diagnose/fix transitions**:
|
|
103
205
|
`omx state write --input '{"mode":"ultraqa","current_phase":"diagnose"}' --json`
|
|
104
206
|
`omx state write --input '{"mode":"ultraqa","current_phase":"fix"}' --json`
|
|
207
|
+
- **On cleanup transition**:
|
|
208
|
+
`omx state write --input '{"mode":"ultraqa","current_phase":"cleanup"}' --json`
|
|
105
209
|
- **On completion**:
|
|
106
210
|
`omx state write --input '{"mode":"ultraqa","active":false,"current_phase":"complete","completed_at":"<now>"}' --json`
|
|
107
211
|
- **For resume detection**:
|
|
@@ -109,23 +213,33 @@ Use the CLI-first state surface (`omx state ... --json`) for UltraQA lifecycle s
|
|
|
109
213
|
|
|
110
214
|
## Scenario Examples
|
|
111
215
|
|
|
112
|
-
**Good:** The user says `continue` after the workflow already has a clear next step. Continue the current branch of work
|
|
216
|
+
**Good:** The user says `continue` after the workflow already has a clear next step. Continue the current branch of work, rerun the relevant adversarial scenario, and update the report instead of restarting discovery.
|
|
113
217
|
|
|
114
218
|
**Good:** The user changes only the output shape or downstream delivery step (for example `make a PR`). Preserve earlier non-conflicting workflow constraints and apply the update locally.
|
|
115
219
|
|
|
220
|
+
**Good:** A CLI prints `SUCCESS` while exiting 1. Mark the misleading success output scenario failed, fix the parser or reporting path, and rerun the generated harness.
|
|
221
|
+
|
|
222
|
+
**Bad:** The workflow runs only `npm test`, `npm run build`, `npm run lint`, or `npm run typecheck`, sees green output, and declares UltraQA complete without adversarial dynamic e2e coverage.
|
|
223
|
+
|
|
224
|
+
**Bad:** A generated harness leaves untracked files, state, or a child process behind and the final report omits cleanup status.
|
|
225
|
+
|
|
116
226
|
**Bad:** The user says `continue`, and the workflow restarts discovery or stops before the missing verification/evidence is gathered.
|
|
117
227
|
|
|
118
228
|
## Cancellation
|
|
119
229
|
|
|
120
|
-
User can cancel with `/cancel
|
|
230
|
+
User can cancel with `/cancel`, which clears UltraQA state. Cancellation itself should be tested in cancel/resume scenarios when relevant, but UltraQA must not block an explicit user cancellation.
|
|
121
231
|
|
|
122
232
|
## Important Rules
|
|
123
233
|
|
|
124
|
-
1. **
|
|
125
|
-
2. **
|
|
126
|
-
3. **
|
|
127
|
-
4. **
|
|
128
|
-
5. **
|
|
234
|
+
1. **ADVERSARIAL E2E REQUIRED** - Baseline build/lint/typecheck/test commands are necessary evidence, not sufficient completion proof.
|
|
235
|
+
2. **SCENARIO MATRIX REQUIRED** - Track normal, hostile, malformed, interruption, injection, cancel/resume, stale-state, dirty-worktree, hung-command, flaky, and misleading-output coverage.
|
|
236
|
+
3. **GENERATE HARNESSES WHEN USEFUL** - Create temporary tests or harnesses when they materially improve behavioral confidence, then clean them up or commit them intentionally.
|
|
237
|
+
4. **PARALLEL WHEN SAFE** - Run independent diagnostics while preparing potential fixes; do not parallelize commands that mutate the same state or worktree.
|
|
238
|
+
5. **TRACK FAILURES** - Record each failure to detect patterns and avoid false greens.
|
|
239
|
+
6. **EARLY EXIT ON PATTERN** - 3x same failure = stop and surface with root cause and residual risk.
|
|
240
|
+
7. **CLEAR OUTPUT** - User should always know current cycle, scenario, command, status, and evidence.
|
|
241
|
+
8. **CLEAN UP** - Clear UltraQA state and temporary artifacts on completion, cancellation, or early stop.
|
|
242
|
+
9. **SAFETY FIRST** - Never exfiltrate secrets, run destructive cleanup, write to production, or wait indefinitely to satisfy a scenario.
|
|
129
243
|
|
|
130
244
|
## STATE CLEANUP ON COMPLETION
|
|
131
245
|
|
|
@@ -133,8 +247,8 @@ When goal is met OR max cycles reached OR exiting early, run `$cancel` or call:
|
|
|
133
247
|
|
|
134
248
|
`omx state clear --input '{"mode":"ultraqa"}' --json`
|
|
135
249
|
|
|
136
|
-
Use CLI state cleanup rather than deleting files directly.
|
|
250
|
+
Use CLI state cleanup rather than deleting files directly. Also remove temporary e2e harnesses, fixtures, and logs unless they are intentional artifacts listed in the report.
|
|
137
251
|
|
|
138
252
|
---
|
|
139
253
|
|
|
140
|
-
Begin ULTRAQA cycling now. Parse the goal and start cycle 1.
|
|
254
|
+
Begin ULTRAQA cycling now. Parse the goal, build the adversarial dynamic e2e scenario matrix, and start cycle 1.
|
|
@@ -27,7 +27,7 @@ This is an orchestration skill. It composes existing skills and must not add run
|
|
|
27
27
|
|
|
28
28
|
## Do not use when
|
|
29
29
|
|
|
30
|
-
- The user only wants design
|
|
30
|
+
- The user only wants repo-wide design guidance, product/design context, or a DESIGN.md source of truth; use `$design` or a designer lane.
|
|
31
31
|
- The task is a non-visual backend/API implementation with no UI reference target.
|
|
32
32
|
- The user already supplied a final static reference image and only needs comparison/fixes; hand directly to `$ralph` with Visual Ralph verdict guidance.
|
|
33
33
|
- The requested output is a deterministic SVG/vector/code-native asset rather than a raster reference.
|
|
@@ -117,7 +117,7 @@ Record final diff evidence with the reference/screenshot artifacts so the result
|
|
|
117
117
|
|
|
118
118
|
### 7. Build a reproducible design system
|
|
119
119
|
|
|
120
|
-
The implementation is incomplete unless the visual match is encoded in repo-native reusable artifacts. Depending on the project, this may mean CSS variables, theme tokens, Tailwind config, component variants, Storybook stories,
|
|
120
|
+
The implementation is incomplete unless the visual match is encoded in repo-native reusable artifacts. Depending on the project, this may mean CSS variables, theme tokens, Tailwind config, component variants, Storybook stories, updates that align with DESIGN.md, or existing equivalents.
|
|
121
121
|
|
|
122
122
|
Capture at least the applicable:
|
|
123
123
|
- colors,
|