create-pmos 0.5.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +114 -0
- package/bin/create-pmos.js +93 -7
- package/package.json +7 -3
- package/payload/AGENTS-HUMAN.md +6 -0
- package/payload/AGENTS.md +35 -1
- package/payload/KIT-VERSION +2 -2
- package/payload/README.md +6 -0
- package/payload/SECURITY.md +6 -0
- package/payload/okf/core/concepts/frontmatter-format.md +75 -0
- package/payload/okf/core/concepts/index.md +3 -0
- package/payload/okf/core/concepts/log.md +13 -0
- package/payload/okf/core/templates/eval-rubric-template.md +12 -10
- package/payload/okf/core/templates/prd-template.md +6 -0
- package/payload/okf/core/templates/prototype-brief-template.md +45 -21
- package/payload/planning/evals/agent-run-eval-rubric.md +10 -7
- package/payload/scripts/metrics.py +142 -121
- package/payload/skills/SKILL-FORMAT.md +6 -0
- package/payload/skills/discovery.skill +4 -0
- package/payload/skills/eval-rubric.skill +1 -1
- package/payload/skills/explain-surface.skill +156 -0
- package/payload/skills/pmos-router.skill +25 -3
- package/payload/skills/prd.skill +8 -0
- package/payload/skills/prototype.skill +106 -41
- package/payload/skills-native/discovery/SKILL.md +4 -0
- package/payload/skills-native/eval-rubric/SKILL.md +1 -1
- package/payload/skills-native/explain-surface/SKILL.md +158 -0
- package/payload/skills-native/pmos-router/SKILL.md +25 -3
- package/payload/skills-native/prd/SKILL.md +8 -0
- package/payload/skills-native/prototype/SKILL.md +106 -41
- package/payload/templates/agent-run-eval-rubric.md +10 -7
- package/payload/templates/ticket-template.md +2 -0
- package/workspace/README.md +112 -0
- package/workspace/REMOTE.md +131 -0
- package/workspace/bin/pmos.js +206 -0
- package/workspace/doctor.js +99 -0
- package/workspace/lib/console-state.js +117 -0
- package/workspace/lib/file-mode-parse.js +58 -0
- package/workspace/lib/harness-parse.js +222 -0
- package/workspace/lib/preview-kind.js +98 -0
- package/workspace/lib/rate-limit.js +26 -0
- package/workspace/lib/remote-access.js +235 -0
- package/workspace/lib/review-gate.js +39 -0
- package/workspace/lib/schedule-logic.js +132 -0
- package/workspace/lib/skill-store.js +143 -0
- package/workspace/lib/surface-contract.js +79 -0
- package/workspace/lib/transcript-read.js +247 -0
- package/workspace/package-lock.json +94 -0
- package/workspace/package.json +23 -0
- package/workspace/public/app.js +1710 -0
- package/workspace/public/index.html +175 -0
- package/workspace/public/styles.css +766 -0
- package/workspace/server.js +1753 -0
- package/workspace/skills/backlog-triage.skill +13 -0
- package/workspace/skills/weekly-report.skill +19 -0
|
@@ -23,7 +23,7 @@ router is the bug (fix it via the weekly harness scan).
|
|
|
23
23
|
> **Format note:** three-level progressive disclosure ([SKILL-FORMAT](/skills/core/SKILL-FORMAT.md)).
|
|
24
24
|
> Sources of truth: the AGENTS.md lane list, `.github/scripts/initiative-gate.sh`, D56/D58/D60.
|
|
25
25
|
|
|
26
|
-
## Step 0 —
|
|
26
|
+
## Step 0 — three checks before any route
|
|
27
27
|
|
|
28
28
|
1. **Sharpness (the fog/ticket test):** *can the PM state the question or idea precisely now — not
|
|
29
29
|
answer it, state it?* If not, it is **fog**: not ready for intake. Name what would sharpen it
|
|
@@ -31,15 +31,37 @@ router is the bug (fix it via the weekly harness scan).
|
|
|
31
31
|
2. **Prior rejection:** check [planning/out-of-scope/](/planning/out-of-scope/README.md) for the
|
|
32
32
|
concept. If a file exists, **read it first** and surface it to the PM — the concept returns only
|
|
33
33
|
if the PM overrides the recorded reasoning, and the file is updated either way.
|
|
34
|
+
3. **The specimen test — ASK it, do not wait to be told** (D60 as amended 2026-07-24):
|
|
35
|
+
|
|
36
|
+
> **Can the PM judge this from a description, or do they need an instance of it in front of them?**
|
|
37
|
+
|
|
38
|
+
Put it to the PM in their own terms for this piece of work. If no description settles it, the
|
|
39
|
+
route is **prototype**, *whether or not the work has a user interface* — the lane is no longer
|
|
40
|
+
UI-gated. This check is asked here because the recorded failure is discoverability: a PM who does
|
|
41
|
+
not know the lane exists cannot ask for it, and a step left to memory decays to never.
|
|
42
|
+
|
|
43
|
+
**It must discriminate in both directions.** Most feature work — an MCP tool, a migration, a CI
|
|
44
|
+
gate — *is* judgeable from a description and routes straight to its PRD untaxed. Over-firing twice
|
|
45
|
+
running is a D41 tripwire on the test's wording.
|
|
46
|
+
|
|
47
|
+
4. **Existing-surface check — does the PM understand what is already there?** Ask:
|
|
48
|
+
|
|
49
|
+
> **Would specifying this honestly require reading code that already exists?**
|
|
50
|
+
|
|
51
|
+
Yes → an [explain-surface](/build-skills/explain-surface.skill) pass runs **first**, before the
|
|
52
|
+
spike brief / prototype brief / PRD is authored, in a fresh context. Greenfield → skip it. This is
|
|
53
|
+
standard, not optional: the PRD skill's own rule ("look it up rather than asking") means the
|
|
54
|
+
*agent* acquires the understanding and the PM does not, and a spec built on facts the PM never saw
|
|
55
|
+
is a watermelon at the root.
|
|
34
56
|
|
|
35
57
|
## The routes
|
|
36
58
|
|
|
37
59
|
| The PM says… | Route | Lane / artifacts / next skill | Gates to expect |
|
|
38
60
|
|---|---|---|---|
|
|
39
|
-
| "I have an idea"
|
|
61
|
+
| "I have an idea" the PM **can't judge from a description** — they need an **instance** in front of them (a UI not yet seen; equally *does this skill's output teach?*, *does this API feel right to call?*) | **Prototype first — before any PRD** | `type: prototype`; quick brief at `prd_path` declaring **the judgement exercised + the specimen format** (**no rubric BY DESIGN** — grilled, never scored); [prototype skill](/skills/core/prototype.skill). Format fits the work: lookable UI, a worked sample output, call/response transcripts, a schema sketch. **The class's conventions are a BLOCKING precondition** (design system for UI; SKILL-FORMAT, D19 ACI, migration/RLS conventions elsewhere) — absent → the agent authors them from the same brief, PM approves, first. | Initiative Gate (brief exists); web-ci quarantine; PM disposition (promote/iterate/kill) |
|
|
40
62
|
| A **new bet** — new product/capability/direction whose *whether* is unargued | **Offer the strategic brief** (recommended, skippable — D61) | [strategic-brief skill](/skills/core/strategic-brief.skill) → `planning/briefs/<id>-brief.md`; go → feature route below; no-go → file in out-of-scope/ | none (the brief is never a gate) |
|
|
41
63
|
| A **validated direction or known feature** (incl. a promoted prototype) | **Feature lane** | `type: feature`; PRD + eval rubric ([prd](/skills/core/prd.skill) + [eval-rubric](/skills/core/eval-rubric.skill) skills), rubric authored alongside (D13); contract before build (D16). A promoted prototype's PRD is **derived from the validated option**. New surface → leg B: [design-brief](/skills/core/design-brief.skill) as the build's reference, held to the [design-reference rubric](/planning/evals/design-reference-eval-rubric.md) | Initiative Gate (PRD+rubric); full three-tier gate |
|
|
42
|
-
| "**Is this feasible / should we / how would we…?**" —
|
|
64
|
+
| "**Is this feasible / should we / how would we…?**" — a **question** with an unknown answer, findable by investigation | **Discovery lane** | `type: discovery`; spike brief at `prd_path` ([discovery skill](/skills/core/discovery.skill)); question + timebox + disposition. Deliverable is a **learning**, and it closes on the **timebox** ("inconclusive" is a recordable outcome). Contrast the row above: that one holds a **candidate** whose rightness no description settles, delivers a **validated direction**, and closes on a **disposition against a specimen**. Widening this lane to cover that was considered and **rejected** in D60 | Initiative Gate (spike brief) |
|
|
43
65
|
| "**Production is broken**" | **Incident lane** | `type: incident`; fix now, record post-hoc within 48h at `prd_path` (the one sanctioned inversion; [incident template](/okf/core/templates/incident-record-template.md)) | Initiative Gate (record, post-hoc) |
|
|
44
66
|
| A **text deliverable** (landing page, microcopy, release notes, campaign) | **Content lane** | `type: content`; content brief at `prd_path`, rubric recommended not required (D58); [copywriting skill](/skills/domains/copywriting.skill). Copy for a prototype rides the prototype instead (D60) | Initiative Gate (content brief) |
|
|
45
67
|
| **Upkeep** — refactor/ops/security with no honest KR | **Chore lane, health-anchored** | `type: chore`; one-paragraph intent in `status`; anchor `health:<id>` from [health-budgets](/planning/okrs/health-budgets.md) (D59) — never a faked KR | Initiative Gate (intent + anchor) |
|
|
@@ -32,6 +32,13 @@ agreed.
|
|
|
32
32
|
|
|
33
33
|
## Procedure
|
|
34
34
|
|
|
35
|
+
0a. **Existing surface? Understand it first — before grilling.** If specifying this honestly
|
|
36
|
+
requires reading code that already exists, an [explain-surface](/build-skills/explain-surface.skill)
|
|
37
|
+
pass runs on that region **first**, in a fresh context and **not by you**. Its *Open questions*
|
|
38
|
+
section is a declared input to the grilling below: those are the things that could not be
|
|
39
|
+
determined from the code, and they must be resolved with the PM or carried explicitly into the
|
|
40
|
+
PRD's risks — never silently answered by assumption. Greenfield → skip. This exists because step 0b's
|
|
41
|
+
boundary rule, on its own, gives the **agent** the understanding and the PM none.
|
|
35
42
|
0. **Grill before you write.** Before any authoring, interrogate the PM's intent as a dialogue —
|
|
36
43
|
**one question at a time, a recommended answer offered with each**, no authoring until shared
|
|
37
44
|
understanding is confirmed. Stop when the intent is **unambiguous**, not at a count — there is
|
|
@@ -74,6 +81,7 @@ agreed.
|
|
|
74
81
|
|
|
75
82
|
A complete PRD instance contains:
|
|
76
83
|
|
|
84
|
+
- **Frontmatter** — the file opens with OKF YAML frontmatter: `type: PRD`, `title`, `timestamp` (the repo-wide convention, D63; [frontmatter-format](/okf/core/concepts/frontmatter-format.md)).
|
|
77
85
|
- **Identity** — `initiative_id`, `product_slug`, `pm_slug`, parent `KR`.
|
|
78
86
|
- **Problem** — who has it, why now, what it costs to leave unsolved.
|
|
79
87
|
- **Goal & non-goals** — the change sought; explicit out-of-scope.
|
|
@@ -1,68 +1,126 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: prototype
|
|
3
|
-
description: Run a pre-PRD
|
|
3
|
+
description: Run a pre-PRD validation through the prototype lane (D60) — the PM gives a QUICK brief, an agent produces 2–3 divergent specimens in whatever format fits the work (lookable UI options, a worked sample output, call/response transcripts, a schema sketch with example rows), a grilling sharpens them (dialogue, never a score), and the PM disposes promote/iterate/kill. Use when an idea's rightness cannot be settled by a description and the PM needs an INSTANCE in front of them — whether or not the work has a user interface; requires the artifact class's conventions (blocking — author them first if absent).
|
|
4
4
|
license: MIT
|
|
5
5
|
metadata:
|
|
6
|
-
pmos-version: 1.
|
|
6
|
+
pmos-version: 1.1.0
|
|
7
7
|
pmos-owner: wawan
|
|
8
8
|
pmos-risk: low
|
|
9
9
|
pmos-category: design
|
|
10
10
|
pmos-scope: read:okf, write:prototype, write:planning
|
|
11
11
|
---
|
|
12
12
|
|
|
13
|
-
# prototype — Run a leg-A
|
|
13
|
+
# prototype — Run a leg-A validation through the prototype lane
|
|
14
14
|
|
|
15
15
|
The [D60](/okf/products/pmos/adr/d60-prototype-loop.md) lane skill. The division of labour is the
|
|
16
|
-
point: **the PM briefs; the agent
|
|
17
|
-
|
|
16
|
+
point: **the PM briefs; the agent builds the specimen.** The PM should never have to author a design
|
|
17
|
+
system, a convention set, or a prototype to validate an idea.
|
|
18
18
|
|
|
19
19
|
> **Format note:** three-level progressive disclosure ([SKILL-FORMAT](/skills/core/SKILL-FORMAT.md)).
|
|
20
20
|
> Level 1 above is the trigger; this body is Level 2.
|
|
21
|
+
> **Scope note (2026-07-24 amendment):** this lane is **no longer gated on the work being a user
|
|
22
|
+
> interface**. D60's mechanism — *make it examinable, then grill it, before committing to a spec* —
|
|
23
|
+
> is general; it was only ever *expressed* visually. The visual path below is unchanged and stays
|
|
24
|
+
> first-class; it is now one shape among several.
|
|
21
25
|
|
|
22
26
|
## When to use vs. not
|
|
23
27
|
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
+
**The entry rule is the specimen test:**
|
|
29
|
+
|
|
30
|
+
> **Can the PM judge this from a description, or do they need an instance of it in front of them?**
|
|
31
|
+
|
|
32
|
+
- **Use** when no description settles it. The measured cost of skipping this is the
|
|
33
|
+
console-redesign series: four post-build rejections that a lookable artifact would have caught
|
|
34
|
+
pre-spec — a *visual* judgement that could only be made once the thing existed. The class is wider
|
|
35
|
+
than the instance: *does this skill's output actually teach?*, *does this API feel right to call?*,
|
|
36
|
+
*does this rubric actually discriminate?*, *does this error message actually help?* are the same
|
|
37
|
+
judgement in a different medium.
|
|
38
|
+
- **Do not use** when a description does settle it — which is most feature work. An MCP tool, a
|
|
39
|
+
migration, a CI gate: the PM can read what it will do and decide. Those go straight to their PRD.
|
|
40
|
+
**A lane that fires on everything has failed as surely as one that fires on nothing**; over-firing
|
|
41
|
+
twice running is a D41 tripwire on the test's wording.
|
|
28
42
|
- **Do not use** for leg B (the post-PRD, high-fidelity design reference — that is the
|
|
29
|
-
[design loop](/okf/core/concepts/design-loop.md): design-brief → design → reconcile,
|
|
30
|
-
|
|
31
|
-
- **Do not use**
|
|
32
|
-
[discovery lane](/skills/core/discovery.skill)
|
|
43
|
+
[design loop](/okf/core/concepts/design-loop.md): design-brief → design → reconcile, open to new
|
|
44
|
+
surfaces per D60's D48 amendment).
|
|
45
|
+
- **Do not use** when you hold a *question* rather than a *candidate* — that is the
|
|
46
|
+
[discovery lane](/skills/core/discovery.skill):
|
|
47
|
+
|
|
48
|
+
| | You hold | Deliverable | Closes on |
|
|
49
|
+
|---|---|---|---|
|
|
50
|
+
| **discovery** | a **question** with an unknown answer, findable by investigation | a **learning** | the timebox — "inconclusive" is a recordable outcome |
|
|
51
|
+
| **prototype** | a **candidate** whose rightness no description settles | a **validated direction** | a disposition against a specimen |
|
|
52
|
+
|
|
53
|
+
Widening `discovery` to cover this was considered and **rejected** in D60 and stays rejected:
|
|
54
|
+
"discovery" is not the word a PM reaches for when they want to *see* something, and
|
|
55
|
+
discoverability was the whole failure.
|
|
33
56
|
|
|
34
57
|
## Procedure
|
|
35
58
|
|
|
36
59
|
1. **Open the lane.** `create_initiative({id, title, parent_kr, type:'prototype', stage:'intake'})` —
|
|
37
60
|
anchored like all work (D21/D59). Branch `initiative/<id>`.
|
|
61
|
+
1b. **Improving something that already exists? Understand it first.** If the specimen will replace
|
|
62
|
+
or alter an existing surface, run an [explain-surface](/build-skills/explain-surface.skill) pass on
|
|
63
|
+
it — fresh context, **before** the brief. Divergent options against a surface nobody has
|
|
64
|
+
characterised diverge from a guess, and the PM cannot grill them honestly. Brand-new → skip.
|
|
38
65
|
2. **Take the quick brief.** The PM fills the
|
|
39
|
-
[prototype-brief template](/okf/core/templates/prototype-brief-template.md) —
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
66
|
+
[prototype-brief template](/okf/core/templates/prototype-brief-template.md) — minutes not hours —
|
|
67
|
+
into `prototype/<id>/brief.md`; set it as `prd_path`. It declares **the judgement being
|
|
68
|
+
exercised** and **the specimen format chosen** (step 4), because those are what the grilling and
|
|
69
|
+
the disposition hang on. **`rubric_path` stays null by design** (grilled, never scored). If the PM
|
|
70
|
+
starts writing requirements, stop them: that detail belongs in the PRD that *follows*.
|
|
71
|
+
3. **Precondition check — BLOCKING (D60(c), generalised).** A specimen built against no shared
|
|
72
|
+
vocabulary produces something nobody can consistently build from. Before producing anything, confirm
|
|
73
|
+
the artifact class has **conventions** and **craft knowledge** to reason from:
|
|
74
|
+
|
|
75
|
+
| Class | Its conventions live in |
|
|
76
|
+
|---|---|
|
|
77
|
+
| A user interface | the product's **design system** — tokens, type scale, components (PMOS's own app: [D38](/DECISIONS.md); tenants checked at onboarding) |
|
|
78
|
+
| A skill | [SKILL-FORMAT](/skills/core/SKILL-FORMAT.md) — three-level progressive disclosure |
|
|
79
|
+
| A tool or API | [D19](/okf/products/pmos/adr/d19-aci-design.md)'s ACI principle |
|
|
80
|
+
| A knowledge doc | the OKF spec + [frontmatter-format](/okf/core/concepts/frontmatter-format.md) |
|
|
81
|
+
| A schema | the repo's migration + RLS conventions |
|
|
82
|
+
|
|
83
|
+
In PMOS most already exist, so this usually passes on inspection. **Where a class has neither,
|
|
84
|
+
authoring them is the first initiative** — propose a minimal set, get the PM's approval, record it
|
|
85
|
+
in the product's namespace, and only then prototype. Never silently invent per-prototype
|
|
86
|
+
conventions; that is how inconsistent products happen.
|
|
87
|
+
4. **Retrieve the craft, and choose the format.** `get_okf_concepts` the class's craft — for a user
|
|
88
|
+
interface, `core/concepts/design-craft` (perception/cognition evidence, craft moves, accessibility
|
|
89
|
+
floors, recorded anti-patterns); for other classes, the conventions named in step 3. **Reason from
|
|
90
|
+
principles to THIS brief; two briefs must not produce the same specimen.** D60(e) is explicit that
|
|
91
|
+
without a craft layer to reason from, an agent told to "apply practice" produces generic output —
|
|
92
|
+
if you cannot name what you are reasoning from, you are in the blocking case above.
|
|
93
|
+
|
|
94
|
+
Then choose the format by the principle — **the cheapest specimen that lets the PM exercise the
|
|
95
|
+
judgement in question**. A starting vocabulary, deliberately **not a closed enum**; an unlisted
|
|
96
|
+
shape takes whatever actually fits, named in the brief:
|
|
97
|
+
|
|
98
|
+
| Work shape | A specimen might be |
|
|
99
|
+
|---|---|
|
|
100
|
+
| A user interface | a low-fidelity lookable artifact (static HTML is usually right) |
|
|
101
|
+
| A skill / agent capability | a worked sample output on real input |
|
|
102
|
+
| A tool or API shape | sample call + response transcripts |
|
|
103
|
+
| A data model | a schema sketch with example rows |
|
|
104
|
+
| An eval rubric | the rubric scored against 2–3 real past artifacts |
|
|
105
|
+
| Error text, CLI, protocol | the actual strings, or a session transcript |
|
|
106
|
+
|
|
107
|
+
5. **Produce divergent specimens** into `prototype/<id>/` — low-fidelity, self-contained, examinable
|
|
108
|
+
(whatever renders or reads fastest). **2–3 genuinely different directions, not one safe one with
|
|
109
|
+
variants** — divergence is the point in any medium. **Real draft content, never placeholder**
|
|
110
|
+
(D60: copy rides the prototype — [copywriting.skill](/skills/domains/copywriting.skill) craft
|
|
111
|
+
applies in-loop; claim-sourcing waits for the content lane at production). The non-visual analogue
|
|
112
|
+
is the same rule: real inputs and real outputs, because fake ones hide exactly the friction the
|
|
113
|
+
specimen exists to surface. Respect the accessibility floors even at low fidelity.
|
|
114
|
+
6. **Grill — dialogue, never a score.** A craft-grade critic (fresh context, default-to-refute
|
|
115
|
+
stance, the class's craft loaded) interrogates each specimen: *why this structure? what job does
|
|
116
|
+
the focal element serve? what would the best practitioner in this class say is wrong? what was NOT
|
|
117
|
+
tried?* One question at a time, a recommended answer with each, no action until shared
|
|
118
|
+
understanding is confirmed. Sharpen the specimens from it. **Produces no score, no verdict, no
|
|
119
|
+
threshold, no `evals` row** — re-introducing scoring here under any name is a D60 violation.
|
|
62
120
|
7. **PM disposition** (the only gate on leg A, and it is the PM's):
|
|
63
121
|
- **promote** → open a **new feature-lane initiative**; its PRD and rubric are **derived from the
|
|
64
|
-
validated
|
|
65
|
-
inside that initiative's Build.
|
|
122
|
+
validated specimen** (what was seen and chosen becomes the spec's substance). Leg B, where the
|
|
123
|
+
work has a surface, then runs inside that initiative's Build.
|
|
66
124
|
- **iterate** → back to step 5 with the PM's reaction folded in.
|
|
67
125
|
- **kill** → record one honest line on the initiative, **file the concept into
|
|
68
126
|
[planning/out-of-scope/](/planning/out-of-scope/README.md)** (decision, reasoning, what would
|
|
@@ -73,10 +131,17 @@ build a prototype to validate an idea.
|
|
|
73
131
|
## Hard rules
|
|
74
132
|
|
|
75
133
|
- Artifacts live in **`prototype/<id>/` only** — never `web/`, never a tenant app. The `web-ci`
|
|
76
|
-
quarantine assertion enforces this;
|
|
134
|
+
quarantine assertion enforces this for the visual case; it is the *instance* of D01's general rule,
|
|
135
|
+
which binds every medium. Do not fight it, and do not read "the assertion only covers `web/`" as
|
|
136
|
+
permission to let a non-visual specimen leak into product code.
|
|
77
137
|
- **A prototype never ships.** Promote produces a *spec*, not a deploy. Quality bars are suspended in
|
|
78
138
|
`prototype/` precisely because nothing there can reach production (D01).
|
|
79
|
-
- **No score on leg A, no skipped grilling on leg A.** Both PM rules bind; neither may be satisfied
|
|
80
|
-
by dropping the other
|
|
139
|
+
- **No score on leg A, no skipped grilling on leg A.** Both PM rules bind; **neither may be satisfied
|
|
140
|
+
by dropping the other**. This holds for non-visual specimens too, where the temptation is sharper:
|
|
141
|
+
a schema or an API *looks* like it has objective criteria, and that is exactly the argument that
|
|
142
|
+
produces safe, scoreable output — the criteria you can score are the ones you already know to look
|
|
143
|
+
for. A technically correct schema can still be the wrong one to live with, and *fit* is what leg A
|
|
144
|
+
is for. (The console log's twin: the evaluator graded *correct* design; the PM judged *desirable*
|
|
145
|
+
design.)
|
|
81
146
|
- If the PM stops looking and promotes on autopilot, say so — an unreviewed disposition is the
|
|
82
147
|
[meta-watermelon](/okf/core/concepts/watermelon-flag.md) in a new place.
|
|
@@ -1,3 +1,13 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Rubric
|
|
3
|
+
title: "Eval Rubric: Agent-run capability baseline (kit seed — copy per initiative)"
|
|
4
|
+
timestamp: 2026-07-19
|
|
5
|
+
eval_type: capability
|
|
6
|
+
initiative_id: (set per initiative)
|
|
7
|
+
pass_threshold: 0.80
|
|
8
|
+
canonical_path: pmos/planning/evals/agent-run-eval-rubric.md
|
|
9
|
+
---
|
|
10
|
+
|
|
1
11
|
# Eval Rubric: Agent-run capability baseline (kit seed — copy per initiative)
|
|
2
12
|
|
|
3
13
|
> Kit-seeded generic rubric (D54). Two purposes: (1) it is the TEMPLATE to copy to
|
|
@@ -6,13 +16,6 @@
|
|
|
6
16
|
> meaningful from day one. It scores a generic agent run outcome-only (D14) with weighted partial
|
|
7
17
|
> credit (D15).
|
|
8
18
|
|
|
9
|
-
## Frontmatter
|
|
10
|
-
- type: Rubric
|
|
11
|
-
- eval_type: capability
|
|
12
|
-
- initiative_id: (set per initiative)
|
|
13
|
-
- pass_threshold: 0.80
|
|
14
|
-
- canonical_path: pmos/planning/evals/agent-run-eval-rubric.md
|
|
15
|
-
|
|
16
19
|
## Gate assignment
|
|
17
20
|
- **Quality Gate (deterministic):** repo CI green; `pmos-gate` (anchored + rubrics) green.
|
|
18
21
|
- **Review Gate (adversarial):** the weighted dimensions below — evaluator skill, default-to-refute,
|
|
@@ -0,0 +1,112 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Reference
|
|
3
|
+
title: "PMOS Terminal Workspace"
|
|
4
|
+
timestamp: 2026-07-25
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# PMOS Terminal Workspace
|
|
8
|
+
|
|
9
|
+
The local companion that runs the PM's agent sessions as **real terminals**, with the PMOS
|
|
10
|
+
workflow (library, gates, needs-you) around them. Built to D62 (as ratified + twice amended):
|
|
11
|
+
a **persistent, localhost-only session server** owning real PTYs, with a **GUI client** attached
|
|
12
|
+
over WebSocket. The native TUI is a deferred later client (D62 amendment 2).
|
|
13
|
+
|
|
14
|
+
## Why it can't lose your harness
|
|
15
|
+
Each session spawns your agent CLI (Claude Code / Codex / a shell) as a real PTY; xterm.js is a
|
|
16
|
+
true terminal emulator on the other end. The CLI runs **untouched** — its own login (Claude
|
|
17
|
+
subscription included), slash commands, keybindings. The workspace is the terminal, not a wrapper.
|
|
18
|
+
|
|
19
|
+
## Everything survives a session
|
|
20
|
+
The server holds PTYs in memory and mirrors each session's scrollback + metadata to `state/`.
|
|
21
|
+
Close/reload the client → re-attach replays scrollback and the run continues (detach, not kill).
|
|
22
|
+
Restart the server → prior sessions reload as `exited` with full history and one-click respawn.
|
|
23
|
+
|
|
24
|
+
## Run it — just type `pmos`
|
|
25
|
+
One-time setup (installs the `pmos` command on your machine):
|
|
26
|
+
```bash
|
|
27
|
+
cd workspace
|
|
28
|
+
npm install -g .
|
|
29
|
+
```
|
|
30
|
+
Then, from any terminal, any folder:
|
|
31
|
+
```bash
|
|
32
|
+
pmos # starts the workspace (installs deps on first run) + opens the browser
|
|
33
|
+
```
|
|
34
|
+
That's it. The server keeps running after you close the terminal — sessions survive (R7).
|
|
35
|
+
```bash
|
|
36
|
+
pmos status # is it running? which sessions?
|
|
37
|
+
pmos stop # shut it down (scrollback is preserved; sessions respawn next time)
|
|
38
|
+
pmos doctor # diagnose the environment (node-pty arch, PTY spawn test)
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
<details><summary>Without the global install (running from the repo)</summary>
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
cd workspace
|
|
45
|
+
npm install # node-pty (native), ws, @xterm/*
|
|
46
|
+
npm start # http://127.0.0.1:7433 (localhost only — never public, D62)
|
|
47
|
+
npm test # end-to-end checks: PTY round-trip, restart-persistence, gates, launcher
|
|
48
|
+
npm run test:ui # browser checks: terminal geometry, reader scroll, markdown (needs Chrome)
|
|
49
|
+
```
|
|
50
|
+
</details>
|
|
51
|
+
|
|
52
|
+
## Layout (PM-directed, D62 R9 + round 2)
|
|
53
|
+
Three columns. **Left = the workspace**: repos on top, running agents below, equal halves.
|
|
54
|
+
**Middle = the terminal**, sessions as tabs. **Right rail = the library over the document
|
|
55
|
+
reader** — either pane folds away (click its header), the whole rail hides (`rail`), and both
|
|
56
|
+
the split and the rail width are draggable. The layout you choose survives a reload.
|
|
57
|
+
|
|
58
|
+
- **Add a repo by picking a folder** — `+` opens your machine's own folder chooser (typing a
|
|
59
|
+
path still works; where no chooser exists, an in-app folder browser stands in).
|
|
60
|
+
- **Rename a session any time** — the ✎ on its row, a double-click on its tab, or `F2`. Live or
|
|
61
|
+
exited, and it survives a server restart.
|
|
62
|
+
- **Documents read like documents** — the identity block becomes a card, tables are tables,
|
|
63
|
+
nested/ordered/task lists survive, and a relative `.md` link opens that document in the reader.
|
|
64
|
+
- **Preview more than markdown** — `.skill` files render as markdown; text/config files open as
|
|
65
|
+
escaped source; `.html` opens as source with an opt-in **sandboxed** render (scripts + network
|
|
66
|
+
denied); markdown renders **repo-relative images inline**. A remote `` image is
|
|
67
|
+
never fetched (no zero-click beacon); a binary stays listed but inert (open it in your editor).
|
|
68
|
+
|
|
69
|
+
## The gates are real, never automatic (R5)
|
|
70
|
+
`approve` / `return` / `accept` write the PMOS file-mode vocabulary to `acceptances.jsonl` and
|
|
71
|
+
clear the matching needs entry — only ever on your click.
|
|
72
|
+
|
|
73
|
+
**The record lands in the repo that earned it.** Each workspace resolves its own state dir —
|
|
74
|
+
`pmos/state/` (the kit's layout, D54), then `state/`, then `.pmos/`; the home workspace uses the
|
|
75
|
+
server's own `state/`. A PMOS-shaped repo with no state dir yet gets the kit layout on first
|
|
76
|
+
write. That path matters: `scripts/metrics.py` reads `pmos/state/acceptances.jsonl` for the north
|
|
77
|
+
star and κ, so an acceptance recorded anywhere else is a green click over an empty record. A
|
|
78
|
+
plain (non-PMOS) repo has no layout to invent — the gate records into the server's `state/` and
|
|
79
|
+
the response says `fallback: true` rather than guessing.
|
|
80
|
+
|
|
81
|
+
Records carry the kit's `accepted` field (`accept` → true, `return` → false) so file-mode metrics
|
|
82
|
+
count them. `pm_label` is **never** derived from a click — κ labels are the PM's own word (D40).
|
|
83
|
+
|
|
84
|
+
## Your own runs are searchable (the history lens)
|
|
85
|
+
Every session's scrollback is already mirrored to disk, which makes it a record of what earlier
|
|
86
|
+
runs actually **did** — the thing structured run rows don't carry. The rail's fourth lens,
|
|
87
|
+
**history**, searches it; the filter box becomes the query. An agent working in a pane can ask
|
|
88
|
+
the same question itself: `GET /api/search?q=…&ws=…`.
|
|
89
|
+
|
|
90
|
+
Two guarantees. It is scoped to **one repo** — a workspace never searches another's history
|
|
91
|
+
(R1/R3), and an unrecognised workspace id resolves to home, so it can only ever see less.
|
|
92
|
+
And every returned line is **scrubbed** of ANSI and of anything credential-shaped (bearer
|
|
93
|
+
tokens, JWTs, `Authorization:` values, api keys): scrollback holds whatever the agent printed,
|
|
94
|
+
so D08's never-log-headers discipline applies to what this hands back. Matching runs against
|
|
95
|
+
the *scrubbed* text, so the endpoint cannot be used to grep a secret out.
|
|
96
|
+
|
|
97
|
+
## Boundaries (honest v1 seams)
|
|
98
|
+
- **Server binds 127.0.0.1** — the D62 hosted-PTY-bridge rejection stands; a tailnet/phone client
|
|
99
|
+
attaches to the same local server, never a public endpoint.
|
|
100
|
+
- **needs-you feed** — an explicit `needs.json` (written by the harness/MCP) is authoritative for
|
|
101
|
+
the entries it names; everything else is **derived** from the repo's own append-only state, so a
|
|
102
|
+
repo nobody has seeded still shows its real work. Derived: an eval with no acceptance for its run
|
|
103
|
+
(the Acceptance Gate is open, D12), and a declared outcome past its horizon with no disposition
|
|
104
|
+
(the D57 check is due — a *missing* horizon counts as due, mirroring `metrics.py`). Derivation
|
|
105
|
+
only reports: an overdue outcome carries **no** close button, because disposal is
|
|
106
|
+
`record_outcome_check` with evidence, and auto-disposal is prohibited (D57).
|
|
107
|
+
- File reads are **path-guarded** to the repo. The reader serves previewable **text kinds**
|
|
108
|
+
(markdown incl. `.skill`, text/config, HTML source) and streams repo-relative **images** through
|
|
109
|
+
a MIME-allowlisted `/api/asset`; binaries and out-of-root/cross-workspace paths are refused.
|
|
110
|
+
Remote images are never fetched (no beacon). Request headers are never logged (D08).
|
|
111
|
+
|
|
112
|
+
Screenshots: `docs/screenshot-dark.png`, `docs/screenshot-light.png`, `docs/screenshot-doc.png`.
|
|
@@ -0,0 +1,131 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Reference
|
|
3
|
+
title: "Workspace remote access over a private tailnet (Tailscale)"
|
|
4
|
+
timestamp: 2026-07-26
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Reaching the workspace from your phone — over a private tailnet, never the public internet
|
|
8
|
+
|
|
9
|
+
The workspace server owns **real terminal sessions (PTYs)** running your `claude` on **your own
|
|
10
|
+
subscription**. By default it binds `127.0.0.1` only, so it is reachable solely from the machine it
|
|
11
|
+
runs on. This runbook enables reaching **the same local server** from another of your own devices —
|
|
12
|
+
a phone, a laptop — over a **private Tailscale tailnet**. It never exposes a public endpoint.
|
|
13
|
+
|
|
14
|
+
## Why a tailnet, and why no public endpoint (D62)
|
|
15
|
+
|
|
16
|
+
A [Tailscale](https://tailscale.com) tailnet is a private [WireGuard](https://www.wireguard.com/)
|
|
17
|
+
mesh between **your own devices**. Every device is authenticated and encrypted end-to-end; addresses
|
|
18
|
+
live in the CGNAT range `100.64.0.0/10` and are routable **only** between devices on your tailnet —
|
|
19
|
+
never from the public internet.
|
|
20
|
+
|
|
21
|
+
PMOS's locked decision **D62** rejects, permanently:
|
|
22
|
+
|
|
23
|
+
- **any public endpoint** for the workspace server, and
|
|
24
|
+
- **any hosted PTY bridge** (proxying the terminal through a hosted, browser-reachable surface).
|
|
25
|
+
|
|
26
|
+
A public server that owns a PTY running your Claude subscription is severe exposure. The tailnet path
|
|
27
|
+
sidesteps that entirely: the phone attaches to the *same local server* across your own private
|
|
28
|
+
network. Accordingly, this feature will **only ever** bind loopback plus your machine's own tailnet
|
|
29
|
+
address. If you ask it to bind anything else (`0.0.0.0`, a LAN or public IP), it **refuses to start**.
|
|
30
|
+
|
|
31
|
+
## Setup
|
|
32
|
+
|
|
33
|
+
1. **Install Tailscale** on the machine running the workspace and on your phone:
|
|
34
|
+
- Machine: https://tailscale.com/download — then `tailscale up`.
|
|
35
|
+
- Phone: install the Tailscale app from your app store and sign in to the **same** tailnet.
|
|
36
|
+
- Confirm the machine's tailnet IPv4: `tailscale ip -4` (a `100.x.y.z` address).
|
|
37
|
+
|
|
38
|
+
2. **Start the server in remote mode**, from `workspace/`:
|
|
39
|
+
|
|
40
|
+
```sh
|
|
41
|
+
PMOS_REMOTE=tailnet npm start
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
On startup it prints (once):
|
|
45
|
+
|
|
46
|
+
```
|
|
47
|
+
pmos workspace server — REMOTE (tailnet) mode
|
|
48
|
+
local: http://127.0.0.1:7433
|
|
49
|
+
tailnet: http://100.x.y.z:7433 (private WireGuard net, device-authenticated, never public — D62)
|
|
50
|
+
token (generated — shown ONCE, copy it now): <a long token>
|
|
51
|
+
attach with header Authorization: Bearer <token> or query ?token=<token>
|
|
52
|
+
loopback requests are token-free; every tailnet request must present the token.
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
**Copy the token now** — it is shown only once and never logged again.
|
|
56
|
+
|
|
57
|
+
3. **Open it from your phone.** In the phone's browser (on the same tailnet), open:
|
|
58
|
+
|
|
59
|
+
```
|
|
60
|
+
http://100.x.y.z:7433/?token=<token>
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
The client attaches to the **same live sessions** the desktop sees; the token authorizes both the
|
|
64
|
+
page and its WebSocket. Requests from the machine itself (`127.0.0.1`) never need the token.
|
|
65
|
+
|
|
66
|
+
## Configuration
|
|
67
|
+
|
|
68
|
+
| Variable | Effect |
|
|
69
|
+
|---|---|
|
|
70
|
+
| `PMOS_REMOTE=tailnet` | Opt in. Bind loopback **+** the machine's tailnet IPv4, and require a token beyond loopback. |
|
|
71
|
+
| `PMOS_BIND_TAILNET=1` | Equivalent to `PMOS_REMOTE=tailnet`. |
|
|
72
|
+
| `PMOS_TOKEN=<t>` | Use your own token instead of a generated one. **Never printed** — supply it yourself on both ends. |
|
|
73
|
+
| `PMOS_TAILNET_IP=<100.x.y.z>` | Pin the tailnet address (skips auto-detection). Must be within `100.64.0.0/10`. |
|
|
74
|
+
| `PMOS_BIND_ADDR=<addr>` | Force a specific bind. Accepted **only** if loopback or within `100.64.0.0/10`; anything else refuses to start. |
|
|
75
|
+
| *(none of the above)* | **Default:** loopback only, no token — unchanged local behavior. |
|
|
76
|
+
|
|
77
|
+
## Security model (what is guaranteed)
|
|
78
|
+
|
|
79
|
+
- **Never public.** The only addresses the server will ever bind are loopback and a
|
|
80
|
+
`100.64.0.0/10` tailnet address. `0.0.0.0`, `::`, LAN, and public IPs are refused — the server
|
|
81
|
+
exits with a clear error rather than binding them. There is no code path to a public bind.
|
|
82
|
+
- **Token beyond loopback.** With remote mode on, every non-loopback HTTP request and WebSocket
|
|
83
|
+
upgrade must present the bearer token, or it is rejected (`401` / socket closed). Loopback requests
|
|
84
|
+
stay token-free for local development. The comparison is constant-time.
|
|
85
|
+
- **Origin/Host allowlist (D64).** Both the HTTP guard and the WebSocket-upgrade handshake reject any
|
|
86
|
+
request whose `Origin` is foreign (not same-origin with the `Host` it arrived on) or whose `Host`
|
|
87
|
+
hostname is not loopback/localhost or your tailnet address. This closes the cross-origin-localhost /
|
|
88
|
+
DNS-rebinding vector: without it, **any website open in your browser** could open a WebSocket to
|
|
89
|
+
`ws://127.0.0.1:<port>` and spawn a shell + type into it, because loopback peers are token-exempt.
|
|
90
|
+
A same-origin page served by the workspace passes; a cross-site page is rejected `403`. Native
|
|
91
|
+
clients / `curl` (which send no `Origin`) still pass this check and are then gated by the token.
|
|
92
|
+
- **No header/token logging (D08).** The request-guard path logs nothing about the request. The
|
|
93
|
+
token is printed at most once at startup (only when generated); a token you supply is never echoed.
|
|
94
|
+
- **Rate-limited control channel (D64).** The WS messages that spawn shells (`new`/`respawn`) and
|
|
95
|
+
feed a PTY (`input`) are per-connection rate-limited, bounding abuse if the socket is ever reached.
|
|
96
|
+
- **Independent layers.** The tailnet (WireGuard device auth), the bearer token, and the origin/host
|
|
97
|
+
allowlist all gate access — a device must be *on your tailnet*, *hold the token*, **and** present a
|
|
98
|
+
same-origin request from an allowed host.
|
|
99
|
+
|
|
100
|
+
## Exposing to the internet — the required hardening (D64)
|
|
101
|
+
|
|
102
|
+
The workspace is **tailnet-first**. The tailnet path above is the supported way to reach it from
|
|
103
|
+
another of your own devices and needs no internet exposure. If you nonetheless expose an instance to
|
|
104
|
+
the internet, D64 permits it **only after all three of the following are in place — never before, and
|
|
105
|
+
never as a raw public bind**:
|
|
106
|
+
|
|
107
|
+
1. **The bearer token for every non-loopback peer** (already enforced in remote mode).
|
|
108
|
+
2. **The Origin/Host allowlist** (already enforced — see above).
|
|
109
|
+
3. **TLS termination at the exposure boundary.** **The workspace server does not terminate TLS.** It
|
|
110
|
+
speaks plain HTTP/WS and must sit behind something that terminates TLS for it — Tailscale's own
|
|
111
|
+
HTTPS (`tailscale serve` / `tailscale funnel`) or a reverse proxy (Caddy, nginx, a tunnel) that
|
|
112
|
+
holds the certificate and forwards to the loopback/tailnet listener. Do **not** put the raw HTTP
|
|
113
|
+
port on a public interface — the server refuses non-loopback/non-tailnet binds precisely so this
|
|
114
|
+
stays a conscious proxy decision, not an accident.
|
|
115
|
+
|
|
116
|
+
When a reverse proxy forwards to the workspace, keep the forwarded `Host` set to the loopback/tailnet
|
|
117
|
+
host the server actually binds, so the Origin/Host allowlist still recognises same-origin requests.
|
|
118
|
+
|
|
119
|
+
## Troubleshooting
|
|
120
|
+
|
|
121
|
+
- **"refusing to bind … only loopback or a Tailscale tailnet address (100.64.0.0/10) is permitted"**
|
|
122
|
+
— you passed `PMOS_BIND_ADDR` (or `PMOS_TAILNET_IP`) that isn't loopback or tailnet. Use your real
|
|
123
|
+
`tailscale ip -4` value.
|
|
124
|
+
- **"no Tailscale tailnet address … was found"** — Tailscale isn't up on this machine. Run
|
|
125
|
+
`tailscale up`, or set `PMOS_TAILNET_IP`.
|
|
126
|
+
- **"cannot bind 100.x.y.z:PORT — that address is not present on this machine"** — the pinned
|
|
127
|
+
`PMOS_TAILNET_IP` isn't actually assigned to this machine. Check `tailscale ip -4`.
|
|
128
|
+
- **Phone gets 401** — the token is missing or wrong. Re-open with `?token=<token>` (or send
|
|
129
|
+
`Authorization: Bearer <token>`). Restarting the server without `PMOS_TOKEN` mints a new token.
|
|
130
|
+
- **Phone can't connect at all** — confirm both devices show each other in the Tailscale app, and
|
|
131
|
+
that you used the `100.x.y.z` address, not a LAN IP.
|