create-pmos 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +30 -0
- package/bin/create-pmos.js +103 -0
- package/package.json +26 -0
- package/payload/AGENTS.md +83 -0
- package/payload/KIT-VERSION +3 -0
- package/payload/README.md +56 -0
- package/payload/okf/core/concepts/agent-harness.md +83 -0
- package/payload/okf/core/concepts/control-plane.md +61 -0
- package/payload/okf/core/concepts/design-loop.md +117 -0
- package/payload/okf/core/concepts/eval-driven-pm.md +71 -0
- package/payload/okf/core/concepts/index.md +17 -0
- package/payload/okf/core/concepts/log.md +6 -0
- package/payload/okf/core/concepts/okf-governance.md +159 -0
- package/payload/okf/core/concepts/okr-anti-optimism.md +54 -0
- package/payload/okf/core/concepts/okr-tree.md +67 -0
- package/payload/okf/core/concepts/output-eval.md +81 -0
- package/payload/okf/core/concepts/pm-identity.md +68 -0
- package/payload/okf/core/concepts/product-slug-convention.md +64 -0
- package/payload/okf/core/concepts/sdlc-loop.md +69 -0
- package/payload/okf/core/concepts/self-refining-harness.md +77 -0
- package/payload/okf/core/concepts/watermelon-flag.md +66 -0
- package/payload/okf/core/templates/design-brief-template.md +84 -0
- package/payload/okf/core/templates/eval-rubric-template.md +95 -0
- package/payload/okf/core/templates/index.md +8 -0
- package/payload/okf/core/templates/log.md +9 -0
- package/payload/okf/core/templates/prd-template.md +75 -0
- package/payload/okf/product/.gitkeep +0 -0
- package/payload/planning/evals/agent-run-eval-rubric.md +46 -0
- package/payload/planning/evals/check_rubrics.py +108 -0
- package/payload/planning/evals/eval-calibration.md +4 -0
- package/payload/planning/okrs/.gitkeep +0 -0
- package/payload/planning/prd/.gitkeep +0 -0
- package/payload/scripts/gate-initiative.sh +56 -0
- package/payload/scripts/kit-update.sh +16 -0
- package/payload/scripts/metrics.py +99 -0
- package/payload/skills/SKILL-FORMAT.md +89 -0
- package/payload/skills/eval-rubric.skill +122 -0
- package/payload/skills/evaluator.skill +82 -0
- package/payload/skills/ingest-repo.skill +117 -0
- package/payload/skills/okr.skill +68 -0
- package/payload/skills/prd.skill +80 -0
- package/payload/state/acceptances.jsonl +0 -0
- package/payload/state/discoveries.md +2 -0
- package/payload/state/evals.jsonl +0 -0
- package/payload/state/initiatives/.gitkeep +0 -0
- package/payload/state/runs.jsonl +0 -0
- package/payload/templates/agent-run-eval-rubric.md +46 -0
- package/payload/templates/pmos-gate.yml +29 -0
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Concept
|
|
3
|
+
title: Eval-Driven Product Management
|
|
4
|
+
description: PMOS's core method — write the eval before the build. The rubric is the spec; the PM manages by defining what "good" means, not by inspecting work after the fact.
|
|
5
|
+
tags: [evaluation, method, product-management, eval-driven]
|
|
6
|
+
timestamp: 2026-06-28
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# Eval-Driven Product Management
|
|
10
|
+
|
|
11
|
+
**Situating context:** This concept names PMOS's central operating method — the thing that makes it a
|
|
12
|
+
distinct way to do product management rather than just a folder of skills. It ties together the
|
|
13
|
+
[SDLC loop](/okf/core/concepts/sdlc-loop.md), the [eval gate](/okf/core/concepts/output-eval.md), and
|
|
14
|
+
[pre-run contracts](/okf/core/concepts/sdlc-loop.md). It informs why the eval-rubric skill exists and
|
|
15
|
+
why contracts come before builds.
|
|
16
|
+
|
|
17
|
+
## The core move: write the eval first
|
|
18
|
+
|
|
19
|
+
Eval-driven PM is test-driven development lifted to the product altitude: **define how you will judge
|
|
20
|
+
the output before any output exists.** The eval rubric is written, and the pre-run contract agreed,
|
|
21
|
+
*before* the agent builds anything.
|
|
22
|
+
|
|
23
|
+
This inverts the default. The default is: build, then ask "is this good?" Eval-driven PM is: decide
|
|
24
|
+
what "good" means, then build to it. The rubric **is** the spec; building is the act of satisfying a
|
|
25
|
+
spec that was made measurable in advance.
|
|
26
|
+
|
|
27
|
+
## Why before, not after
|
|
28
|
+
|
|
29
|
+
Writing the eval after the build corrupts the judgment in three ways:
|
|
30
|
+
|
|
31
|
+
1. **The output anchors the standard.** Once you have seen what the agent produced, your idea of
|
|
32
|
+
"good enough" silently bends to match it. A rubric written in ignorance of the output is the only
|
|
33
|
+
honest one.
|
|
34
|
+
2. **There is nothing to fail against.** Without a prior standard, evaluation degenerates into "does
|
|
35
|
+
this look fine?" — which is exactly the [evaluator leniency](/okf/core/concepts/output-eval.md)
|
|
36
|
+
failure, made structural.
|
|
37
|
+
3. **Drift has no tripwire.** A pre-committed standard is what lets you notice
|
|
38
|
+
[silent drift](/okf/core/concepts/watermelon-flag.md): the output meets the rubric or it does
|
|
39
|
+
not, and you defined the rubric back when you still remembered the intent.
|
|
40
|
+
|
|
41
|
+
## The PM manages by defining "good," not by inspecting work
|
|
42
|
+
|
|
43
|
+
In eval-driven PM, the PM's primary leverage is *upstream*: authoring the rubric, agreeing the
|
|
44
|
+
contract, setting the threshold. By the time work comes back, most of the management has already
|
|
45
|
+
happened — it lives in the standard the work is measured against.
|
|
46
|
+
|
|
47
|
+
This is why the PM's end-of-run role is the [Acceptance Gate](/okf/core/concepts/output-eval.md), not
|
|
48
|
+
line-by-line inspection. The Quality and Review gates check the work against the pre-written rubric
|
|
49
|
+
mechanically; the PM's scarce attention is spent on the one question the rubric cannot answer — *is
|
|
50
|
+
this the right thing?* — and on whether the rubric itself was right.
|
|
51
|
+
|
|
52
|
+
## How it shapes the loop
|
|
53
|
+
|
|
54
|
+
- **Spec stage** authors the rubric alongside the PRD — the
|
|
55
|
+
[eval-rubric skill](/skills/core/eval-rubric.skill) and [prd skill](/skills/core/prd.skill) are
|
|
56
|
+
paired for this reason.
|
|
57
|
+
- **Contract stage** turns the rubric into agreed, testable done-criteria the generator and evaluator
|
|
58
|
+
both sign off on before work starts.
|
|
59
|
+
- **Evaluate stage** runs the output against the pre-committed rubric — outcome-only, weighted,
|
|
60
|
+
partial-credit.
|
|
61
|
+
- **Learn stage** feeds rubric calibration back into
|
|
62
|
+
[eval-calibration.md](/planning/evals/eval-calibration.md): when the PM and evaluator disagree, the
|
|
63
|
+
rubric or the evaluator prompt — not the memory of what happened — gets fixed.
|
|
64
|
+
|
|
65
|
+
## Calibration is part of the method
|
|
66
|
+
|
|
67
|
+
An eval-driven system is only as trustworthy as its evals, and PMOS's evals start
|
|
68
|
+
**uncalibrated** (evaluator leniency). Eval-driven PM therefore includes a calibration discipline:
|
|
69
|
+
treat early P1 runs as calibration cycles, log PM overrides, and patch the evaluator until PM and
|
|
70
|
+
evaluator agree. The method is not "trust the eval" — it is "make the eval trustworthy, then manage
|
|
71
|
+
through it."
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# core/concepts — Index
|
|
2
|
+
|
|
3
|
+
OKF reserved file (no frontmatter). Directory listing for progressive disclosure — read this
|
|
4
|
+
to pick concepts, then fetch the file(s) you need.
|
|
5
|
+
|
|
6
|
+
- [agent-harness](agent-harness.md) — Everything wrapping the model that lets it finish work — and the Summary Gate, the input gate that compresses prior-session context into essential state.
|
|
7
|
+
- [control-plane](control-plane.md) — What it means for PMOS to be a control plane — it orchestrates, specs, evaluates, and routes agent work rather than generating production code itself.
|
|
8
|
+
- [design-loop](design-loop.md) — How PMOS makes UI/UX design a first-class, repeatable sub-flow of the Build stage — brief → design → reconcile → build → design-sync — gated by the Review Gate, presentation-only, and never a data decision.
|
|
9
|
+
- [eval-driven-pm](eval-driven-pm.md) — PMOS's core method — write the eval before the build. The rubric is the spec; the PM manages by defining what "good" means, not by inspecting work after the fact.
|
|
10
|
+
- [okf-governance](okf-governance.md) — How PMOS organizes its Open Knowledge Format bundle and the governance rules that keep it spec-conformant.
|
|
11
|
+
- [okr-anti-optimism](okr-anti-optimism.md) — Why PMOS biases status reporting toward pessimism — surfacing the red underneath green metrics — to keep KR self-reporting honest.
|
|
12
|
+
- [okr-tree](okr-tree.md) — How PMOS structures Objectives and Key Results as a tree that links strategy to initiatives and to the agent runs that move them.
|
|
13
|
+
- [output-eval](output-eval.md) — How PMOS evaluates agent output through three escalating gates — Quality, Review, Acceptance — and why the PM owns only the last one.
|
|
14
|
+
- [pm-identity](pm-identity.md) — How PMOS models a product manager — the pm_slug identifier, the PM's authority as the human in the loop, and PM-scoped namespaces.
|
|
15
|
+
- [product-slug-convention](product-slug-convention.md) — The rules for product_slug — the stable lowercase identifier that scopes products across OKF namespaces, planning artifacts, and agent records.
|
|
16
|
+
- [sdlc-loop](sdlc-loop.md) — The closed agent-driven software development lifecycle PMOS orchestrates — spec, contract, build, evaluate, accept, learn — and how it loops.
|
|
17
|
+
- [watermelon-flag](watermelon-flag.md) — A status that reads green on automated metrics but is red underneath on strategic alignment — and silent drift, the L4 failure mode it signals.
|
|
@@ -0,0 +1,159 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Concept
|
|
3
|
+
title: OKF Namespace Structure & Governance Model
|
|
4
|
+
description: How PMOS organizes its Open Knowledge Format bundle and the governance rules that keep it spec-conformant.
|
|
5
|
+
tags: [okf, governance, knowledge-management, namespace, reference-layer]
|
|
6
|
+
timestamp: 2026-06-29
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# OKF Namespace Structure & Governance Model
|
|
10
|
+
|
|
11
|
+
**Situating context:** This is the first OKF concept authored in PMOS. It exists because every
|
|
12
|
+
other concept, template, and playbook in the bundle needs a shared, unambiguous answer to two
|
|
13
|
+
questions: *what is OKF here* and *how do we keep the bundle from drifting into a bespoke schema*.
|
|
14
|
+
It informs every authoring decision for files under `/okf/`.
|
|
15
|
+
|
|
16
|
+
## What OKF is
|
|
17
|
+
|
|
18
|
+
OKF stands for **Open Knowledge Format** — the Google Cloud open spec (v0.1 draft) at
|
|
19
|
+
<https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md>.
|
|
20
|
+
|
|
21
|
+
It is **not** "Organizational Knowledge Fabric" or any other expansion. Do not invent one.
|
|
22
|
+
|
|
23
|
+
OKF is a deliberately minimal format: **a directory of Markdown files with YAML frontmatter.**
|
|
24
|
+
There is no schema registry, no central authority, and no required tooling. PMOS conforms strictly
|
|
25
|
+
to this spec rather than layering a custom schema on top of it.
|
|
26
|
+
|
|
27
|
+
## What lives in OKF (and what does not)
|
|
28
|
+
|
|
29
|
+
OKF holds **timeless, curated knowledge** — the things that stay true across sessions and remain
|
|
30
|
+
useful as reference:
|
|
31
|
+
|
|
32
|
+
- Concepts (mental models, definitions, principles)
|
|
33
|
+
- ADRs (architecture decision records)
|
|
34
|
+
- Templates (PRD template, eval-rubric template)
|
|
35
|
+
- Playbooks (repeatable procedures)
|
|
36
|
+
|
|
37
|
+
OKF does **not** hold operational records. PRDs instances, eval-rubric instances, OKR instances,
|
|
38
|
+
and agent run traces are operational state — they live in Supabase (P1+) and under `/planning/`
|
|
39
|
+
(P0), **not** in the OKF bundle. The test: if it is a dated instance of work, it is operational;
|
|
40
|
+
if it is reusable knowledge, it belongs in OKF.
|
|
41
|
+
|
|
42
|
+
### Two layers: concepts and references
|
|
43
|
+
|
|
44
|
+
The bundle holds two kinds of curated knowledge that age differently:
|
|
45
|
+
|
|
46
|
+
- **Concept layer** (`core/`) — timeless mental models, definitions, principles, templates,
|
|
47
|
+
playbooks. Changes by insight, rarely.
|
|
48
|
+
- **Reference layer** (`products/<product_slug>/`) — a **catalog of the system's concrete assets**:
|
|
49
|
+
one small, individually-retrievable doc per Supabase table (its schema + RLS shape), per derived
|
|
50
|
+
view/metric, per MCP tool (the ACI), and per locked decision (an `ADR` view of `DECISIONS.md`).
|
|
51
|
+
This is exactly how upstream OKF bundles work — they catalog every table, metric, and join — and
|
|
52
|
+
it is the layer that **must track the backbone**: it changes whenever the system changes.
|
|
53
|
+
|
|
54
|
+
Both layers are still *knowledge about the system*, not operational rows. A `data-model` doc
|
|
55
|
+
describes a table's **shape and rules**; it never contains the table's data. An `ADR` doc is a
|
|
56
|
+
curated **view** of a decision; `DECISIONS.md` stays the single source of truth. The reference layer
|
|
57
|
+
is kept in sync by the [okf-sync playbook](/okf/core/playbooks/okf-sync.md) — the standing cadence
|
|
58
|
+
that stops the bundle freezing while the product evolves.
|
|
59
|
+
|
|
60
|
+
## The bundle
|
|
61
|
+
|
|
62
|
+
A **Knowledge Bundle** is a self-contained hierarchical collection of OKF concept documents — the
|
|
63
|
+
unit of distribution. PMOS's `/okf/` tree is one bundle:
|
|
64
|
+
|
|
65
|
+
```
|
|
66
|
+
okf/ ← the bundle root
|
|
67
|
+
├── index.md ← reserved: directory listing (progressive disclosure)
|
|
68
|
+
├── log.md ← reserved: chronological update history, newest first
|
|
69
|
+
├── core/ ← canonical, cross-cutting knowledge
|
|
70
|
+
│ ├── concepts/ ← mental models and definitions (this file lives here)
|
|
71
|
+
│ ├── templates/ ← reusable document templates
|
|
72
|
+
│ └── playbooks/ ← repeatable procedures
|
|
73
|
+
├── products/
|
|
74
|
+
│ └── pmos/ ← product-scoped knowledge namespace
|
|
75
|
+
├── shared/ ← concepts promoted for cross-PM reuse
|
|
76
|
+
└── sources/ ← raw ingested artifacts (immutable)
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
**Concept ID = the file path within the bundle, minus the `.md` suffix.** For example, this
|
|
80
|
+
document's concept ID is `core/concepts/okf-governance`.
|
|
81
|
+
|
|
82
|
+
### Namespace intent
|
|
83
|
+
|
|
84
|
+
- `core/` — knowledge that is canonical to PMOS and cuts across products.
|
|
85
|
+
- `products/<product_slug>/` — knowledge scoped to one product (e.g. `products/pmos/`).
|
|
86
|
+
- `shared/` — concepts promoted out of a product or PM namespace because they generalize across
|
|
87
|
+
PMs. Promotion is deliberate, not automatic.
|
|
88
|
+
- `sources/` — raw ingested artifacts kept **immutable** as provenance. Concepts are extracted
|
|
89
|
+
*from* sources into `core/`/`products/`; sources themselves are never edited in place.
|
|
90
|
+
|
|
91
|
+
## Frontmatter rules (spec-conformant)
|
|
92
|
+
|
|
93
|
+
The **only required** frontmatter field is **`type`** — a short, self-explanatory string. Examples:
|
|
94
|
+
`Concept`, `Playbook`, `ADR`, `Reference`, `Template`.
|
|
95
|
+
|
|
96
|
+
**Recommended** fields, in priority order:
|
|
97
|
+
|
|
98
|
+
1. `title` — human-readable name.
|
|
99
|
+
2. `description` — one-line summary; used for relevance decisions during retrieval.
|
|
100
|
+
3. `resource` — canonical URI for the concept. **Omit** for abstract concepts that have no
|
|
101
|
+
canonical external URI.
|
|
102
|
+
4. `tags` — list of short topic labels.
|
|
103
|
+
5. `timestamp` — ISO 8601 last-modified date.
|
|
104
|
+
|
|
105
|
+
### Fields we explicitly dropped
|
|
106
|
+
|
|
107
|
+
Earlier PMOS design drafts referenced custom required fields: `status`, `confidence`,
|
|
108
|
+
`last_validated`, `namespace`, `owner`, `knowledge_type`, `enrichment_cadence`. **These are not
|
|
109
|
+
required and default to absent.** Conforming to the open spec means not reinventing its schema.
|
|
110
|
+
|
|
111
|
+
- Map `knowledge_type` → spec `type`.
|
|
112
|
+
- Map `last_validated` → spec `timestamp`.
|
|
113
|
+
- If any other field is genuinely needed, it may appear **only** as an optional, producer-defined
|
|
114
|
+
extension key — but the default is to **not** add it.
|
|
115
|
+
|
|
116
|
+
## Reserved filenames
|
|
117
|
+
|
|
118
|
+
Two filenames are reserved and **must not** be used as concept documents:
|
|
119
|
+
|
|
120
|
+
- **`index.md`** — a directory listing that supports progressive disclosure (an agent reads the
|
|
121
|
+
index to decide what else to load). It has **no frontmatter**.
|
|
122
|
+
- **`log.md`** — a chronological update history, **newest first**, using ISO dates.
|
|
123
|
+
|
|
124
|
+
## Cross-linking
|
|
125
|
+
|
|
126
|
+
Link concepts with standard Markdown links. **Absolute bundle-relative links** (beginning with `/`)
|
|
127
|
+
are preferred, e.g. `[OKR tree](/okf/core/concepts/okr-tree.md)`. Broken links are **tolerated** —
|
|
128
|
+
they may point at knowledge that has not been written yet, which is a feature, not an error.
|
|
129
|
+
|
|
130
|
+
## Governance: permissive conformance
|
|
131
|
+
|
|
132
|
+
PMOS governance is intentionally permissive. Tooling and reviewers **must tolerate**:
|
|
133
|
+
|
|
134
|
+
- Unknown `type` values.
|
|
135
|
+
- Missing optional (recommended) fields.
|
|
136
|
+
- Unknown extra producer-defined keys.
|
|
137
|
+
- Broken cross-links.
|
|
138
|
+
- Missing `index.md` files.
|
|
139
|
+
|
|
140
|
+
CI frontmatter linting **may enforce** the presence of the `type` field, but **must not** reject a
|
|
141
|
+
document for missing optional fields or for carrying extra keys. The bias is toward accepting
|
|
142
|
+
knowledge and improving it in place, not gatekeeping it out.
|
|
143
|
+
|
|
144
|
+
## Lifecycle
|
|
145
|
+
|
|
146
|
+
1. **Ingest** — raw artifacts land immutably in `/okf/sources/`.
|
|
147
|
+
2. **Extract** — concepts are distilled from sources into `core/concepts/` or
|
|
148
|
+
`products/<product_slug>/`, each with situating context (where it came from, why, what decision
|
|
149
|
+
it informs).
|
|
150
|
+
3. **Sync** — when the live backbone changes (a new table, view, MCP tool, or decision), the matching
|
|
151
|
+
**reference doc** is authored or refreshed in the same change, per the
|
|
152
|
+
[okf-sync playbook](/okf/core/playbooks/okf-sync.md). This is what keeps the reference layer from
|
|
153
|
+
freezing behind the product.
|
|
154
|
+
4. **Promote** — concepts that generalize across PMs move to `/okf/shared/`.
|
|
155
|
+
5. **Record** — every meaningful change is appended to `/okf/log.md` (newest first), and to the
|
|
156
|
+
namespace `log.md` for product-scoped changes.
|
|
157
|
+
|
|
158
|
+
This lifecycle keeps provenance immutable, knowledge curated, the reference layer current, and the
|
|
159
|
+
bundle spec-conformant.
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Concept
|
|
3
|
+
title: OKR Anti-Optimism Reporting
|
|
4
|
+
description: Why PMOS biases status reporting toward pessimism — surfacing the red underneath green metrics — to keep KR self-reporting honest.
|
|
5
|
+
tags: [okr, reporting, governance, drift, honesty]
|
|
6
|
+
timestamp: 2026-06-28
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# OKR Anti-Optimism Reporting
|
|
10
|
+
|
|
11
|
+
**Situating context:** OKRs are self-reported, and self-reported progress drifts optimistic —
|
|
12
|
+
especially when an agent is grading its own work. This concept defines the anti-optimism stance that
|
|
13
|
+
keeps [OKR](/okf/core/concepts/okr-tree.md) reporting honest. It is one of the two structural
|
|
14
|
+
defenses against [silent drift](/okf/core/concepts/watermelon-flag.md) and it informs how every
|
|
15
|
+
status report in PMOS is written.
|
|
16
|
+
|
|
17
|
+
## The default is optimism; the rule is anti-optimism
|
|
18
|
+
|
|
19
|
+
Left alone, status reporting rounds up. A KR at 60% gets reported as "on track." An evaluator unsure
|
|
20
|
+
of an output scores it generously ([evaluator leniency](/okf/core/concepts/output-eval.md)). An agent
|
|
21
|
+
summarizing its own run emphasizes what worked. Each rounding is small; together they manufacture a
|
|
22
|
+
[watermelon flag](/okf/core/concepts/watermelon-flag.md).
|
|
23
|
+
|
|
24
|
+
PMOS counters with a standing rule: **report the red, not the rounded-up green.** When in doubt,
|
|
25
|
+
report the more pessimistic reading and make the uncertainty explicit.
|
|
26
|
+
|
|
27
|
+
## What anti-optimism reporting requires
|
|
28
|
+
|
|
29
|
+
1. **Lead with what is off-track.** A status report names the red and amber items first; green is the
|
|
30
|
+
short tail, not the headline.
|
|
31
|
+
2. **Confidence, not just status.** "KR at 60%, *low confidence* it reaches target" carries more
|
|
32
|
+
information than "on track." State how sure you are, separately from where you are.
|
|
33
|
+
3. **No green without evidence of strategic alignment.** A metric being green is not permission to
|
|
34
|
+
call the initiative green. Automated-green is necessary, not sufficient — the
|
|
35
|
+
[Acceptance Gate](/okf/core/concepts/output-eval.md) confirms strategic green.
|
|
36
|
+
4. **Name the gap to intent.** Where the work has bent away from the parent objective, say so
|
|
37
|
+
explicitly, even if every individual run passed.
|
|
38
|
+
5. **Surface friction.** Anything that fought the harness goes to
|
|
39
|
+
[discoveries.md](/discoveries.md) tagged `#friction`, not buried in a success summary.
|
|
40
|
+
|
|
41
|
+
## Why this is structural, not cultural
|
|
42
|
+
|
|
43
|
+
"Be honest in reports" is a vague mandate; PMOS makes it a structural requirement because the system
|
|
44
|
+
depends on it. Anti-optimism reporting is what makes a watermelon **visible while it is still green
|
|
45
|
+
on the skin** — early enough for the PM to re-anchor the initiative before accumulated drift becomes
|
|
46
|
+
expensive. Pair it with the Acceptance Gate: the gate is the human check, anti-optimism reporting is
|
|
47
|
+
the input that gives the human something true to check.
|
|
48
|
+
|
|
49
|
+
## Anti-optimism is not pessimism for its own sake
|
|
50
|
+
|
|
51
|
+
The goal is *calibration*, not gloom. Over-reporting red is also a failure — it trains the PM to
|
|
52
|
+
ignore reports. The discipline is: report the honest reading, weight uncertainty toward the
|
|
53
|
+
downside, and always separate "where we are" from "how confident we are." A well-calibrated amber is
|
|
54
|
+
worth more than a confident green or a reflexive red.
|
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Concept
|
|
3
|
+
title: OKR Tree Structure
|
|
4
|
+
description: How PMOS structures Objectives and Key Results as a tree that links strategy to initiatives and to the agent runs that move them.
|
|
5
|
+
tags: [okr, strategy, planning, measurement]
|
|
6
|
+
timestamp: 2026-06-28
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# OKR Tree Structure
|
|
10
|
+
|
|
11
|
+
**Situating context:** PMOS routes agent work, and work without a strategic anchor is how silent
|
|
12
|
+
drift starts. This concept defines the OKR tree so every initiative — and ultimately every agent run
|
|
13
|
+
— can be traced to a measurable objective. It informs the `okr` skill, OKR instance files under
|
|
14
|
+
`/planning/okrs/`, and how the Acceptance Gate judges strategic fit.
|
|
15
|
+
|
|
16
|
+
## The tree
|
|
17
|
+
|
|
18
|
+
PMOS models strategy as a tree, not a flat list:
|
|
19
|
+
|
|
20
|
+
```
|
|
21
|
+
Objective (qualitative, directional — the "what we want to be true")
|
|
22
|
+
└── Key Result (quantitative, measurable — the "how we'll know")
|
|
23
|
+
└── Initiative (a unit of work — owns a PRD + eval rubric)
|
|
24
|
+
└── Agent run (a single execution against the initiative's contract)
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
- **Objective** — a qualitative, time-boxed statement of intent. Inspirational and directional;
|
|
28
|
+
carries no number itself. *"PMOS agent output is trustworthy enough to ship without rework."*
|
|
29
|
+
- **Key Result** — a quantitative, measurable outcome that proves the objective is being met.
|
|
30
|
+
Each objective has 2–5 KRs. *"Human correction rate per agent run < 0.5 by end of Q3."*
|
|
31
|
+
- **Initiative** — a concrete body of work expected to move one or more KRs. Each initiative owns a
|
|
32
|
+
PRD ([prd skill](/skills/core/prd.skill)) and an eval rubric
|
|
33
|
+
([eval-rubric skill](/skills/core/eval-rubric.skill)).
|
|
34
|
+
- **Agent run** — the leaf: one execution against an initiative's pre-run contract, logged to
|
|
35
|
+
`agent_runs`.
|
|
36
|
+
|
|
37
|
+
## Edges carry meaning
|
|
38
|
+
|
|
39
|
+
The value of the tree is in its edges. Every node must answer "what is my parent?":
|
|
40
|
+
|
|
41
|
+
- A KR with no parent objective is an orphan metric — measuring something nobody decided matters.
|
|
42
|
+
- An initiative with no parent KR is **unanchored work** — the structural seed of silent drift.
|
|
43
|
+
PMOS treats unanchored initiatives as a red flag at intake, not a detail to fill in later.
|
|
44
|
+
- An agent run inherits its strategic justification from its initiative's KR. This is what lets the
|
|
45
|
+
Acceptance Gate ask "is this the *right* thing?" with a concrete reference.
|
|
46
|
+
|
|
47
|
+
## Key Results measure outcomes, not output
|
|
48
|
+
|
|
49
|
+
A KR measures a **change in the world**, not a count of work done. "Ship 5 features" is output;
|
|
50
|
+
"reduce human correction rate to < 0.5" is an outcome. Output-based KRs invite the watermelon-flag
|
|
51
|
+
failure: green activity numbers hiding red strategic reality. Pair this concept with
|
|
52
|
+
[anti-optimism reporting](/okf/core/concepts/okr-anti-optimism.md) and the
|
|
53
|
+
[watermelon flag](/okf/core/concepts/watermelon-flag.md) — they are the defenses that keep KR
|
|
54
|
+
self-reporting honest.
|
|
55
|
+
|
|
56
|
+
## The north star sits above the tree
|
|
57
|
+
|
|
58
|
+
PMOS's north-star metric — **human correction rate per agent run** (lower is better) — is the
|
|
59
|
+
standing measure the tree exists to move. Quarterly objectives change; the north star persists.
|
|
60
|
+
Individual KRs should ladder toward it, directly or indirectly.
|
|
61
|
+
|
|
62
|
+
## Cadence
|
|
63
|
+
|
|
64
|
+
- OKRs are set per planning period (PMOS's first set: `okrs-q3-2026.md` under `/planning/okrs/`).
|
|
65
|
+
- KRs are reviewed on the PM's regular cadence; objectives are reset each period.
|
|
66
|
+
- The OKR file is an **operational instance** — it lives under `/planning/`, not in OKF. This
|
|
67
|
+
concept is the timeless *structure*; the quarterly file is the dated *content*.
|
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Concept
|
|
3
|
+
title: Output Evaluation — The Three-Tier Gate
|
|
4
|
+
description: How PMOS evaluates agent output through three escalating gates — Quality, Review, Acceptance — and why the PM owns only the last one.
|
|
5
|
+
tags: [evaluation, gates, quality, acceptance, governance]
|
|
6
|
+
timestamp: 2026-06-28
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# Output Evaluation — The Three-Tier Gate
|
|
10
|
+
|
|
11
|
+
**Situating context:** Every turn of the [SDLC loop](/okf/core/concepts/sdlc-loop.md) ends by
|
|
12
|
+
judging what an agent produced. This concept defines the three-tier gate that judgment runs through,
|
|
13
|
+
so the [eval-rubric skill](/skills/core/eval-rubric.skill), AGENTS.md, and the PM all use the same
|
|
14
|
+
vocabulary. It informs where automation stops and human judgment begins.
|
|
15
|
+
|
|
16
|
+
## The three tiers
|
|
17
|
+
|
|
18
|
+
Output passes through three escalating gates. Each answers a different question, with a different
|
|
19
|
+
nature and a different owner.
|
|
20
|
+
|
|
21
|
+
### 1. Quality Gate — *does it work?*
|
|
22
|
+
|
|
23
|
+
- **Nature:** deterministic, automated.
|
|
24
|
+
- **Checks:** compile, lint, unit tests — toolchain-enforced pass/fail.
|
|
25
|
+
- **Owner:** the toolchain. No judgment involved; either the tests pass or they do not.
|
|
26
|
+
- A failure here never reaches a human; it bounces back to the agent.
|
|
27
|
+
|
|
28
|
+
### 2. Review Gate — *does it satisfy the rubric?*
|
|
29
|
+
|
|
30
|
+
- **Nature:** probabilistic, adversarial.
|
|
31
|
+
- **Checks:** does the output meet the rubric criteria for this initiative? Applied by a separate
|
|
32
|
+
**evaluator agent** or by the PM acting as critic.
|
|
33
|
+
- **Owner:** evaluator agent (or PM-as-critic).
|
|
34
|
+
- This gate is **uncalibrated** until 5–10 P1 cycles are validated — the evaluator defaults to
|
|
35
|
+
generous scoring (**evaluator leniency**). Treat its verdict as advisory until calibrated against
|
|
36
|
+
[eval-calibration.md](/planning/evals/eval-calibration.md).
|
|
37
|
+
|
|
38
|
+
### 3. Acceptance Gate — *is it the right thing?*
|
|
39
|
+
|
|
40
|
+
- **Nature:** subjective, strategic.
|
|
41
|
+
- **Checks:** strategic fit. Is this what the product actually needs right now? Does it move the
|
|
42
|
+
parent KR? **This is NOT a re-check of the rubric.**
|
|
43
|
+
- **Owner:** the **human PM only**. Non-delegable.
|
|
44
|
+
- The PM's role at the end of every agent run **is** the Acceptance Gate.
|
|
45
|
+
|
|
46
|
+
## Why three, and why the order
|
|
47
|
+
|
|
48
|
+
The gates escalate from cheap-and-certain to expensive-and-strategic:
|
|
49
|
+
|
|
50
|
+
- Quality is **cheap and certain** — run it first, fail fast, never spend human attention on broken
|
|
51
|
+
output.
|
|
52
|
+
- Review is **mid-cost and probabilistic** — it filters rubric misses before they reach the PM, but
|
|
53
|
+
its judgment is fallible.
|
|
54
|
+
- Acceptance is **expensive and irreducibly human** — it is the only gate that asks whether the work
|
|
55
|
+
*should* exist, not whether it was done correctly.
|
|
56
|
+
|
|
57
|
+
Running them out of order wastes the scarce resource (PM attention) on problems the cheap gates
|
|
58
|
+
would have caught.
|
|
59
|
+
|
|
60
|
+
## The Acceptance Gate is not the Review Gate
|
|
61
|
+
|
|
62
|
+
The most important distinction in this concept: **passing the rubric is not the same as being
|
|
63
|
+
right.** An output can satisfy every rubric criterion and still be the wrong thing to build — that
|
|
64
|
+
is exactly the [silent drift](/okf/core/concepts/watermelon-flag.md) failure. The Acceptance Gate
|
|
65
|
+
exists *because* rubric satisfaction is not strategic correctness. Collapsing the two — letting a
|
|
66
|
+
green Review Gate auto-accept — removes the only structural defense against drift.
|
|
67
|
+
|
|
68
|
+
## Grading discipline
|
|
69
|
+
|
|
70
|
+
- **Outcome-only:** gates check what the agent *produced*, not the path it took. Do not grade tool
|
|
71
|
+
call sequences.
|
|
72
|
+
- **Weighted, with partial credit:** the Review Gate scores weighted dimensions and totals against a
|
|
73
|
+
threshold; it is not all-or-nothing. (See the
|
|
74
|
+
[eval-rubric skill](/skills/core/eval-rubric.skill) and
|
|
75
|
+
[template](/okf/core/templates/eval-rubric-template.md).)
|
|
76
|
+
|
|
77
|
+
## Relationship to autonomy
|
|
78
|
+
|
|
79
|
+
As PMOS moves L2 → L3, more of the Quality and Review gates run unattended. The Acceptance Gate
|
|
80
|
+
stays human at every autonomy level up to the declared L3 ceiling. Keeping that gate human is what
|
|
81
|
+
makes higher autonomy safe rather than reckless.
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Concept
|
|
3
|
+
title: PM Identity Model
|
|
4
|
+
description: How PMOS models a product manager — the pm_slug identifier, the PM's authority as the human in the loop, and PM-scoped namespaces.
|
|
5
|
+
tags: [identity, pm, governance, namespace]
|
|
6
|
+
timestamp: 2026-06-28
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# PM Identity Model
|
|
10
|
+
|
|
11
|
+
**Situating context:** PMOS is a PM control plane, so "who is the PM" is a primitive the whole
|
|
12
|
+
system is built around — paths, records, and the final human gate all reference it. This concept
|
|
13
|
+
fixes that model so skills, OKF namespaces, and agent records resolve a PM the same way. It informs
|
|
14
|
+
every place a `pm_slug` appears and every decision about who holds authority.
|
|
15
|
+
|
|
16
|
+
## The PM is the human in the loop
|
|
17
|
+
|
|
18
|
+
PMOS does not replace the product manager — it gives the PM a control surface over agent work. The
|
|
19
|
+
PM is the single accountable human. Agents generate; the PM **decides**. This is structural, not
|
|
20
|
+
advisory: the end of every agent run is an **Acceptance Gate** (a subjective, strategic "is this
|
|
21
|
+
the *right* thing?" judgment) that **only the human PM** can clear. See
|
|
22
|
+
[/okf/core/concepts/output-eval.md](/okf/core/concepts/output-eval.md) for the three-tier gate
|
|
23
|
+
model the Acceptance Gate sits in.
|
|
24
|
+
|
|
25
|
+
## `pm_slug` — the stable identifier
|
|
26
|
+
|
|
27
|
+
Every PM has a **`pm_slug`**: a stable, lowercase, URL-safe identifier. It is the canonical
|
|
28
|
+
reference to a PM across paths, records, and skill namespaces.
|
|
29
|
+
|
|
30
|
+
- Rules: lowercase, no spaces, stable for the life of the PM's participation. Treat it like a
|
|
31
|
+
primary key — never rename casually, because paths and records point at it.
|
|
32
|
+
- For this instance, **`pm_slug: wawan`**.
|
|
33
|
+
- It pairs with `product_slug` (see
|
|
34
|
+
[/okf/core/concepts/product-slug-convention.md](/okf/core/concepts/product-slug-convention.md))
|
|
35
|
+
to fully scope work: *which PM*, *which product*.
|
|
36
|
+
|
|
37
|
+
## PM-scoped namespaces
|
|
38
|
+
|
|
39
|
+
The `pm_slug` carves out PM-specific space where a PM can hold variant knowledge and skills without
|
|
40
|
+
disturbing the canonical core:
|
|
41
|
+
|
|
42
|
+
- **`/skills/pm/<pm_slug>/`** — PM-specific skill variants. A PM may override or specialize a core
|
|
43
|
+
skill here (e.g. `/skills/pm/wawan/`). The core skill in `/skills/core/` remains canonical;
|
|
44
|
+
the PM variant is a deliberate, scoped divergence.
|
|
45
|
+
- **PM-promoted concepts** — knowledge a PM develops that generalizes across PMs is promoted into
|
|
46
|
+
[/okf/shared/](/okf/shared/), per the OKF governance lifecycle. Promotion is deliberate; a PM's
|
|
47
|
+
private working knowledge does not auto-publish to `shared/`.
|
|
48
|
+
|
|
49
|
+
## PM authority and responsibilities
|
|
50
|
+
|
|
51
|
+
The PM's authority is bounded and explicit:
|
|
52
|
+
|
|
53
|
+
- **Owns acceptance.** Clears (or rejects) the Acceptance Gate at the end of each run. This is a
|
|
54
|
+
strategic-fit decision, **not** a re-check of the eval rubric.
|
|
55
|
+
- **Owns architectural decisions.** All locked decisions live in `DECISIONS.md`; agents never make
|
|
56
|
+
architectural calls. The PM is the author and amender of those decisions.
|
|
57
|
+
- **Owns calibration.** The PM validates the evaluator against
|
|
58
|
+
[eval-calibration.md](/planning/evals/eval-calibration.md) over the first P1 run cycles, because
|
|
59
|
+
the eval gate is uncalibrated until then (evaluator leniency).
|
|
60
|
+
- **Owns the cadence.** Weekly transcript review, monthly kill-condition checks, quarterly harness
|
|
61
|
+
assumption review — the anti-atrophy mechanisms that defend against the Paradox of Supervision.
|
|
62
|
+
|
|
63
|
+
## Why model identity explicitly
|
|
64
|
+
|
|
65
|
+
Modelling the PM as a first-class, slug-addressable identity is what lets PMOS scope work
|
|
66
|
+
unambiguously (which PM, which product), keep PM-specific variants isolated from canonical core, and
|
|
67
|
+
attribute the north-star metric — **human correction rate per agent run** — to a specific human's
|
|
68
|
+
judgment. Identity is the anchor the control plane hangs accountability on.
|
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
---
|
|
2
|
+
type: Concept
|
|
3
|
+
title: product_slug Convention
|
|
4
|
+
description: The rules for product_slug — the stable lowercase identifier that scopes products across OKF namespaces, planning artifacts, and agent records.
|
|
5
|
+
tags: [identity, product, namespace, convention]
|
|
6
|
+
timestamp: 2026-06-28
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
# product_slug Convention
|
|
10
|
+
|
|
11
|
+
**Situating context:** PMOS is built to run many products through one control plane, so it needs a
|
|
12
|
+
stable way to say "this knowledge / this work belongs to *that* product." This concept fixes the
|
|
13
|
+
`product_slug` convention so paths and records resolve a product unambiguously. It pairs with the
|
|
14
|
+
[PM identity model](/okf/core/concepts/pm-identity.md): together `pm_slug` + `product_slug` fully
|
|
15
|
+
scope any unit of work — which PM, which product.
|
|
16
|
+
|
|
17
|
+
## What `product_slug` is
|
|
18
|
+
|
|
19
|
+
A **`product_slug`** is a stable, lowercase, URL-safe identifier for one product. It is the
|
|
20
|
+
canonical reference to that product everywhere PMOS records or routes work.
|
|
21
|
+
|
|
22
|
+
For this instance: **`product_slug: pmos`** (PMOS is the first product, building itself).
|
|
23
|
+
|
|
24
|
+
## Rules
|
|
25
|
+
|
|
26
|
+
1. **Lowercase only.** No uppercase, no spaces.
|
|
27
|
+
2. **URL- and path-safe.** Letters, digits, hyphens. No slashes, dots, or underscores that would
|
|
28
|
+
collide with path or frontmatter parsing. Prefer hyphens for multi-word slugs.
|
|
29
|
+
3. **Stable.** Treat it as a primary key. Renaming a `product_slug` orphans every path and record
|
|
30
|
+
that points at it, so it changes only by deliberate PM decision with a migration.
|
|
31
|
+
4. **Unique** within the PMOS instance.
|
|
32
|
+
5. **Short and recognizable.** It appears in paths constantly; favour the obvious short form.
|
|
33
|
+
|
|
34
|
+
## Where it appears
|
|
35
|
+
|
|
36
|
+
- **OKF namespace:** `/okf/products/<product_slug>/` holds knowledge scoped to one product
|
|
37
|
+
(e.g. `/okf/products/pmos/`). Core, cross-product knowledge stays in `/okf/core/`; product-scoped
|
|
38
|
+
knowledge lives under its slug.
|
|
39
|
+
- **Planning artifacts:** PRD, OKR, and eval instances under `/planning/` reference the product by
|
|
40
|
+
its slug so each dated artifact resolves to a product.
|
|
41
|
+
- **Agent records:** `agent_runs` and initiative records carry the `product_slug` (P1) so the
|
|
42
|
+
north-star metric can be sliced per product.
|
|
43
|
+
|
|
44
|
+
## Scoping with `pm_slug`
|
|
45
|
+
|
|
46
|
+
`product_slug` answers *which product*; `pm_slug` answers *which PM*. Together they scope work
|
|
47
|
+
completely:
|
|
48
|
+
|
|
49
|
+
- `pm_slug: wawan` + `product_slug: pmos` → Wawan's work on PMOS.
|
|
50
|
+
- This pairing is what lets PMOS attribute outcomes and isolate variant knowledge without ambiguity
|
|
51
|
+
as more PMs and products come online.
|
|
52
|
+
|
|
53
|
+
## Core vs. product-scoped knowledge
|
|
54
|
+
|
|
55
|
+
A recurring authoring decision: does this concept go in `/okf/core/` or
|
|
56
|
+
`/okf/products/<product_slug>/`?
|
|
57
|
+
|
|
58
|
+
- **Core** — true across products; part of the PMOS method itself (e.g. the OKR tree, the eval gate
|
|
59
|
+
model). Most P0 seed concepts are core.
|
|
60
|
+
- **Product-scoped** — true only for one product's domain or constraints. Place under the product's
|
|
61
|
+
slug.
|
|
62
|
+
|
|
63
|
+
When unsure, default to core only if the knowledge would still be correct for a second, unrelated
|
|
64
|
+
product; otherwise scope it to the product.
|