ecoportal-api-graphql 1.3.12 → 1.3.13

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,243 @@
1
+ # Spike — Reverse-Classifying Existing Template Questions into Archetypes
2
+
3
+ > **Question.** Can we read the ~300 existing templates' questions and cluster them back into
4
+ > "categories/archetypes" — where an archetype = *parent question (Select/Checklist) + dependent
5
+ > fields + the force(s) that reveal/require/compute them per answer*? What does that depend on?
6
+ >
7
+ > **Scope.** Feasibility + prerequisites only. No code changes. Companion to
8
+ > `DESIGN_NOTE.md` (which frames the *authoring* direction); this spike is the *reverse* direction.
9
+ > All code claims are cited `file:line` and verified against the current tree
10
+ > (branch `feature/model-input-fixes`, 2026-07-07).
11
+
12
+ ---
13
+
14
+ ## 1. Feasibility verdict
15
+
16
+ **Partially feasible today; fully feasible with one small, already-triaged query-shape fix.**
17
+
18
+ - The **structural half** of an archetype (parent Select/Checklist + its options + adjacent
19
+ dependent fields, plus `genomeSignature` and `ref` as equivalence signals) is **readable now** —
20
+ the page/template read model and its fragments already fetch everything needed.
21
+ - The **flow half** (which answer reveals/requires/computes which field) lives in **Forces**, and
22
+ **reading forces off a page is the one blocked path** — but the blocker is a *client-side
23
+ query-shape bug*, not a missing backend or a missing model. The Force model, bindings model,
24
+ read fragment, and read query all already exist in the gem; the query and fragment simply select
25
+ fields the schema rejects. The fix is enumerated to the line in `.ai-assistance/projects/TODO.md`.
26
+ - Even once forces are readable, the **force `script` is an AngularJS/LISP string** with no
27
+ structured answer→effect representation available today. Extracting `answer == "Yes" → show(F)`
28
+ requires parsing that string. This is the **honest residual risk** and it does not go away with
29
+ the query fix (see §5, §6).
30
+
31
+ So: **structure-only classification is unblocked and gives a useful first pass; authoritative
32
+ classification (with the flow contract) needs (a) the forces-read fix — cheap — and (b) a force
33
+ script parser — the real work.**
34
+
35
+ ---
36
+
37
+ ## 2. Per-signal reliability assessment
38
+
39
+ Ranked by authoritativeness for identifying an archetype.
40
+
41
+ ### 2.1 Force bindings + script — AUTHORITATIVE, but gated + needs parsing
42
+
43
+ A `Force` connects fields/sections via **bindings** and encodes the show/hide/require/compute logic
44
+ in its `script`. This is the *only* signal that actually tells you the flow contract.
45
+
46
+ - **Model exists.** `Base::Force` exposes `id`, `name`, `script`, `customScript`, `weight`, `url`,
47
+ `globalBindingIds`, and `bindings` — `lib/ecoportal/api/graphql/base/force.rb:19-28`.
48
+ - **Bindings expose the field/section link.** `Base::Force::Binding` carries `name` + `referenceId`
49
+ (the bound field/section id) — `lib/ecoportal/api/graphql/base/force/binding.rb:7-14`. The
50
+ collection can look up by name — `.../binding_collection.rb:19-22`.
51
+ - **The page model surfaces forces** — `Interface::BasePage#forces` wraps `doc['forces']` in a
52
+ `Force::Collection`, and the page carries a `hasForces` boolean —
53
+ `lib/ecoportal/api/graphql/interface/base_page.rb:75-83`.
54
+ - **A read fragment + read query exist.** `Fragment::ForceFields` selects
55
+ `id name script customScript globalBindingIds bindings{...}` —
56
+ `lib/ecoportal/api/graphql/fragment/force.rb:6-26`. `Query::PageWithForces` fetches
57
+ `page { ...PageFields; forces { ...ForceFields } }` —
58
+ `lib/ecoportal/api/graphql/query/page_with_forces.rb:31-38`.
59
+
60
+ **Reliability caveats even once readable:**
61
+ - `bindings` give you *which* fields a force touches (the cluster membership) — high value, structured.
62
+ - The `script` gives you the *per-answer effect* — but only as an unparsed AngularJS/LISP string
63
+ (`script` / `customScript` are opaque text). No structured answer→effect map is exposed. See §5.
64
+ - A force applies to a stage only if *all* its bindings resolve in that stage
65
+ (`10_forces_workflow_builder.md:16-21`), so binding→field resolution must be done per stage.
66
+
67
+ ### 2.2 Field structure / adjacency — AVAILABLE NOW, heuristic only
68
+
69
+ "A Select followed by N fields in the same section" is readable directly from the page/template read
70
+ model: stages → sections → fields.
71
+
72
+ - Sections and their fields are fetched by the page fragment —
73
+ `lib/ecoportal/api/graphql/fragment/pages/common_page_union.rb:105-131` (`sectionUnion` →
74
+ `ContentSection.dataFields` / `SplitSection.left/rightDataFields`).
75
+ - The gem exposes stage→section→field traversal (`page.stages`, `section.components`,
76
+ `get_by_type`, `get_by_name`) — documented in `07_data_fields.md:152-168`.
77
+ - 20+ concrete field types are dispatched via `Base::Page::DataField::TYPE_MAP`
78
+ (`07_data_fields.md:19-45`); `Select`/`Checklist` parents are identifiable by `__typename`.
79
+
80
+ **Reliability: heuristic.** Adjacency correlates with, but does not prove, a conditional
81
+ relationship. A detail field can sit adjacent to its parent for layout reasons with no force, or a
82
+ force can reveal a field in a *different* section. Adjacency gives you *candidate* clusters to then
83
+ confirm against forces. On its own it will both over-group (adjacent-but-independent fields) and
84
+ miss (force reveals a distant field). **Usable for a first-pass bucketing; fragile as ground truth.**
85
+
86
+ ### 2.3 genomeSignature — STRONG-BUT-FALLIBLE equivalence signal (not a clustering signal)
87
+
88
+ - **Fetched today.** The data-field fragment selects `genomeSignature`
89
+ (`fragment/pages/common_page_union.rb:124`) and the model reads it —
90
+ `lib/ecoportal/api/graphql/base/page/data_field.rb:45` (`passthrough :label, :deindex, :genomeSignature`).
91
+ - **Already consumed as a pairing signal.** The diff pairing engine uses it, weighted highest
92
+ (0.5) but explicitly not ground truth — `lib/ecoportal/api/graphql/diff/pairing/signals.rb:24-35`
93
+ and `WEIGHTS` at `:17-22`. The header documents 6 known failure modes
94
+ (`signals.rb:6-12`).
95
+
96
+ **Role for classification: equivalence, not clustering.** genomeSignature tells you *"this field in
97
+ template A is the same field as that field in template B"* — invaluable for saying *"the same
98
+ archetype instance recurs across N templates"* once you've identified it once. It does **not** tell
99
+ you *which fields form a cluster within one template* (that's bindings/adjacency). Treat it as the
100
+ cross-template dedup / recurrence signal, not the intra-template grouping signal. Fallible: reused/
101
+ re-purposed fields keep a stale genome, and newer field types may lack one (`signals.rb:8-12,31`).
102
+
103
+ ### 2.4 Select option sets, labels/descriptions, `ref` — SUPPORTING signals
104
+
105
+ - **Option sets** — the parent's option labels/ids are read as part of the field data
106
+ (`Select` typed field; `07_data_fields.md:59`). Useful to name the archetype (e.g. a Yes/No
107
+ reveal) and to map `reveal_on` answers once the script is parsed.
108
+ - **Labels/descriptions** — `label` is fetched (`common_page_union.rb:123`). The DESIGN_NOTE notes
109
+ the platform's identity trick is a hidden section-marker field + field `description` as keys
110
+ (`DESIGN_NOTE.md:96-98`) — so `description` doubles as an authored archetype/section key when present.
111
+ - **`ref`** — the ES index key, `<type>.<hash_of_label>` (`07_data_fields.md:181-197`). Stable-ish
112
+ per label; **breaks when the label changes** and can be per-instance if a field was script-added
113
+ without copying from template (`07_data_fields.md:194-197`). Weak identity aid, not a cluster key.
114
+
115
+ ---
116
+
117
+ ## 3. Is fixing the forces-read blocker a PREREQUISITE?
118
+
119
+ **For authoritative reverse-classification (archetype WITH its flow contract): YES.**
120
+ The flow half of an archetype is defined by forces (bindings + script). Bindings are the structured
121
+ statement of *which fields move together*; without reading them you are guessing cluster membership
122
+ from adjacency alone. There is no other exposed source of the conditional flow on the platform today
123
+ (the Workflow Builder's conditional-field workflows that would replace forces are still *in
124
+ development* — `10_forces_workflow_builder.md:83-96`, `86-88`).
125
+
126
+ **For a structure-only first pass (adjacency + genome + options): NO.**
127
+ You can bucket questions into *candidate* archetypes from the read model alone (§2.2–2.4) without any
128
+ forces access. This is useful for triage and to size the problem, but it is **fragile** (see §2.2):
129
+ it cannot distinguish "Select with a genuinely conditional follow-up" from "Select that happens to
130
+ sit above an always-visible field," and it will miss cross-section reveals. Any number you report
131
+ from adjacency-only is a *lower-confidence estimate*, not a classification.
132
+
133
+ **Net:** the forces-read fix is a **prerequisite for the real deliverable** but **not** for an
134
+ initial scoping pass. Given the fix is cheap (§4), there is little reason to ship the fragile
135
+ version as anything more than a scoping estimate.
136
+
137
+ ---
138
+
139
+ ## 4. Minimal-unblock sketch (forces read)
140
+
141
+ The blocker is a **client query-shape bug**, fully triaged in `.ai-assistance/projects/TODO.md:14-24`
142
+ (verified there against the live schema on 2026-07-03 — "all THREE are REAL bugs"). Two edits:
143
+
144
+ 1. **`Query::PageWithForces` selects `forces` directly on a union.** The query hangs `forces` off
145
+ the `PageUnion`, which is invalid GraphQL (can't select a field on a bare union) —
146
+ `query/page_with_forces.rb:33-35` (`forces { spread :ForceFields }`). **Fix:** wrap in the shared
147
+ interface — `... on BasePageInterface { forces { ...ForceFields } }` (per TODO.md:16-17).
148
+
149
+ 2. **`Fragment::ForceFields` selects `id` on binding types that have none.** The fragment selects
150
+ `id` inside `... on DataFieldBinding` and `... on SectionBinding`
151
+ (`fragment/force.rb:14-23`), but those types expose only `name`/`referenceId` (no `id`) —
152
+ `TODO.md:16-17`. **Fix:** drop `id` from the binding selections (keep `name`, `referenceId`).
153
+ Note `Base::Force::Binding` declares `passkey :id` (`base/force/binding.rb:8`) — harmless once the
154
+ query stops requesting it, since the model tolerates a missing key.
155
+
156
+ 3. **(eco-helpers, sibling repo — not this gem)** Split the capability gate.
157
+ `OozeRedirect.force_support?` hard-returns `false`, which gates **read and write together**, so
158
+ the broken read disables the working write as collateral — `10_forces_workflow_builder.md:50-54`,
159
+ RCA `.../analysis/2026-07-01-forces-via-workflow-commands-miss-rca.md:28-36,59-70`. For
160
+ *reverse-classification* we only need the **read** path, so this repo's job is items 1–2; the
161
+ eco-helpers gate split matters for the ooze scripts, not for a classifier that calls
162
+ `Query::PageWithForces` directly.
163
+
164
+ **Verification step (not code):** these three queries have no render/validation specs; the same TODO
165
+ item asks to add a render + schema-validation spec alongside the fix and to wire
166
+ `tests/validate_queries.rb` into CI (`TODO.md:5-24`). Do that first so the fix is provably
167
+ schema-valid against a fresh introspected schema before relying on it against 300 templates.
168
+
169
+ **Effort:** two one-line selection edits + specs. Small. The model/fragment/query scaffolding is all
170
+ already present (shipped as WIP alongside the write path in v1.3.11 per `CHANGELOG.md:184`).
171
+
172
+ ---
173
+
174
+ ## 5. Parsing the force script — the real difficulty
175
+
176
+ Reading forces gets you the **bindings** (structured: which fields) and the **script** (unstructured:
177
+ what happens per answer). The archetype's *per-answer effect* — `reveal_on: "Yes" → show + require`
178
+ in the DESIGN_NOTE worked example (`DESIGN_NOTE.md:44-56`) — lives **only in the script string**.
179
+
180
+ - `script`/`customScript` are opaque text (`base/force.rb:20`); the gem treats them as strings it
181
+ round-trips, never parses (write path just sets `customScript` and queues `editForce` —
182
+ `base/force.rb:33-38,55-60`). There is **no structured effect representation** exposed anywhere.
183
+ - The scripts are **AngularJS/LISP snippets** (`10_forces_workflow_builder.md:9`,
184
+ `base/force.rb:6`). Extracting a reliable `answer → {show|hide|require|compute}(field)` mapping
185
+ from arbitrary such snippets is an **interpreter/parser problem**, with a long tail of hand-written
186
+ variants across 300 templates and 10+ orgs (~22 force-dependent ooze cases catalogued —
187
+ RCA `:27`). Expect: bespoke idioms, computed risk-matrix math, multi-condition guards, indirection
188
+ through global bindings (`globalBindingIds` — `base/force.rb:22`).
189
+ - The **bindings alone** are a meaningful partial win: they tell you the *cluster membership* (this
190
+ force ties this Select to these N fields/sections) with high confidence, even if you never parse
191
+ the script. That yields "these fields form a conditional cluster governed by force X" — enough to
192
+ identify *that* an archetype exists and its members, if not its exact per-answer semantics.
193
+
194
+ **Structured alternative:** the **Workflow Builder** conditional-field workflows are the intended
195
+ structured replacement for force scripts — but they are **in development, not deployed**
196
+ (`10_forces_workflow_builder.md:83-96`) and the API is explicitly "not stable enough for scripting
197
+ against its internals" (`:98-104`). So there is **no structured flow representation available today**
198
+ for existing templates; existing conditional logic is in AngularJS forces. Reverse-classification of
199
+ the *installed base* cannot wait for Workflow Builder without leaving the ~300 current templates
200
+ uncovered.
201
+
202
+ ---
203
+
204
+ ## 6. Open risks (honest)
205
+
206
+ 1. **AngularJS script parsing (highest).** No structured effect map exists; extracting
207
+ answer→effect from arbitrary LISP/AngularJS snippets across 300 templates is the dominant unknown.
208
+ Mitigate by (a) classifying on **bindings + adjacency + options** first (cluster membership +
209
+ candidate parent), and (b) parsing scripts only for a **closed catalog** of common idioms,
210
+ flagging the long tail as "custom force — unclassified" rather than mis-parsing. This mirrors the
211
+ DESIGN_NOTE's closed-catalog recommendation (`DESIGN_NOTE.md:126-130`).
212
+ 2. **Forces-read fix unproven against a live org.** The fix is triaged from schema introspection but
213
+ the query has never run green against a force-bearing page (it shipped as WIP —
214
+ `TODO.md:24`, RISKS `.../ooze-graphql-native-migration/RISKS.md:19-30`). Live-test
215
+ `Query::PageWithForces` on a real force-bearing page before trusting bulk output.
216
+ 3. **Binding→field resolution is per-stage and can dangle.** A force shows only if all bindings
217
+ resolve in the stage (`10_forces_workflow_builder.md:16-21`); bindings referencing removed/renamed
218
+ fields will dangle. Classifier must tolerate unresolved `referenceId`s.
219
+ 4. **genomeSignature fallibility for recurrence.** Re-purposed fields keep a stale genome and newer
220
+ types may lack one (`signals.rb:8-12,31`) — so "same archetype across templates" via genome will
221
+ have false matches/misses. Confirm with type+label+options (the existing pairing engine already
222
+ does exactly this weighting — `signals.rb:17-22`).
223
+ 5. **`ref` instability.** Label-hash based and per-instance when script-added
224
+ (`07_data_fields.md:194-197`) — do **not** use `ref` as a cluster or identity key; it is at best a
225
+ weak tiebreaker.
226
+ 6. **Forces are being deprecated mid-flight.** New templates use Workflow Builder, old ones use
227
+ forces (`10_forces_workflow_builder.md:61-65`). A classifier built purely on force-reading will
228
+ have blind spots on newer Workflow-Builder templates once conditional workflows deploy — plan for
229
+ a second reader against `PagesWorkflow` when that API stabilises.
230
+
231
+ ---
232
+
233
+ ## 7. Bottom line
234
+
235
+ - **Structure-only classification:** doable now, no gem change — but a fragile first-pass estimate.
236
+ - **Authoritative classification (with flow):** requires the **forces-read fix** (two one-line
237
+ selection edits in `Query::PageWithForces` / `Fragment::ForceFields`, already triaged in
238
+ `TODO.md:14-24`, plus a render/validation spec) — **cheap and low-risk**.
239
+ - **The genuine hard part** is not the read fix; it is **parsing AngularJS force scripts** into
240
+ answer→effect mappings. Bindings give cluster membership cheaply; the per-answer semantics require
241
+ a parser, best scoped to a **closed catalog** of common idioms with a raw-force escape hatch.
242
+ - **genomeSignature** is the cross-template recurrence signal (already wired in the diff pairing
243
+ engine), not the intra-template grouping signal.
@@ -0,0 +1,154 @@
1
+ # Design Note — A Template Model That Holds the Logic
2
+
3
+ > For the team meeting. Goal: *"identify a template model that holds the logic — not just
4
+ > questions and options, but the fields that get unhidden depending on the answer."*
5
+ > Audience: engineers. Keep it skimmable.
6
+
7
+ ---
8
+
9
+ ## 1. The problem
10
+
11
+ Our template representations today capture **structure** — stages, sections, fields, and (for
12
+ Selects/Checklists) their options. What they do **not** capture is the **conditional flow**: the
13
+ fact that answering a parent question a certain way reveals follow-up fields, makes them required,
14
+ or computes a value. On the platform that flow lives in **Forces** (AngularJS snippets bound to a
15
+ stage: show/hide sections/fields, conditional-required, computed values — being replaced by the
16
+ Workflow Builder's "conditional field workflows", still in development). See
17
+ `.ai-assistance/code/ecoPortal_architecture/10_forces_workflow_builder.md`. A template model that
18
+ omits forces omits the actual behaviour of the form. This meeting is about giving the model a
19
+ first-class place to hold that logic.
20
+
21
+ ---
22
+
23
+ ## 2. The category / composite question archetype
24
+
25
+ **Reframe Oscar's "category" as a composite question archetype.** A category is not a field type —
26
+ it is a **pattern one level above the flat field**:
27
+
28
+ ```
29
+ archetype = parent question (usually Select / Checklist)
30
+ + its options
31
+ + the dependent fields it governs
32
+ + the force(s) that reveal / require / compute those fields per answer
33
+ ```
34
+
35
+ The **category name** identifies the pattern; the category **carries the flow contract** — which
36
+ answer unhides which fields, which answers flip a field to required, which answers drive a computed
37
+ value. It is a **higher-order primitive above the flat field level**: a cluster of fields plus the
38
+ force wiring that moves them together, referenced by name and parameters instead of spelled out.
39
+
40
+ ### Worked example — "conditional-detail Select"
41
+
42
+ Author intent (one row / one macro):
43
+
44
+ ```
45
+ category: conditional_detail_select
46
+ parent_label: "Was anyone injured?"
47
+ options: ["No", "Yes"]
48
+ reveal_on: "Yes"
49
+ detail_field: { label: "Describe the injury", type: plainText, required_when_shown: true }
50
+ ```
51
+
52
+ Expands to the flat model:
53
+
54
+ - a `Select` field `Was anyone injured?` with options `No`, `Yes`
55
+ - a `PlainText` field `Describe the injury`, hidden by default
56
+ - a **force**: `when answer == "Yes"` → `show(detail_field)` + `set_required(detail_field)`
57
+
58
+ One category, authored once, produces **two fields + one force with two effects**. Change the
59
+ category and all of that regenerates coherently.
60
+
61
+ ---
62
+
63
+ ## 3. Where it sits in the pipeline
64
+
65
+ The category is a **macro that expands into `fields + forces`** — it lives *above* the flat IR, not
66
+ beside it. Layering:
67
+
68
+ ```
69
+ requirements
70
+
71
+ form design
72
+
73
+ CATEGORIES (composite archetypes: parent + options + dependents + forces) ← the new layer
74
+ ↓ expand
75
+ flat IR (EPTIR: fields + tags + forces) ← existing (parked)
76
+ ↓ compile
77
+ ecoPortal implementation (WorkflowCommandInput command-bus) ← shipped
78
+ ```
79
+
80
+ The flat IR already exists in draft as **EPTIR** (ecoPortal Template Intermediate Representation,
81
+ parked in `C:\ruby_scripts\git\eptir`, v0.1 / `2026.1` DRAFT). The category layer does not replace
82
+ EPTIR — it **compiles down into it**. `Builder::TemplateBuilder` + `Diff::CommandSynthesizer`
83
+ already compile a flat declarative spec into `WorkflowCommandInput` batches, so the bottom two
84
+ layers are real; categories add a compile step on top.
85
+
86
+ ---
87
+
88
+ ## 4. What it buys us
89
+
90
+ - **Compresses the CSV pipeline.** The CSV→~300-templates work
91
+ (`.ai-assistance/projects/api-v2-to-graphql-migration/notes/csv-template-pipeline-design.md`,
92
+ delivery Sept 2026) today needs one row per field plus separate spelling-out of every dependent
93
+ field and force. A category row references `category + params` and expands to the whole cluster —
94
+ far fewer rows, far less force hand-authoring, mistakes localised to the macro.
95
+ - **A natural pairing / identity anchor.** A category is a cluster of fields that move together, so
96
+ it is the obvious identity unit. It rides directly on the existing identity tricks — the
97
+ hidden section-marker field (section key) and field `description` (field key) — by anchoring the
98
+ whole cluster to one category key instead of tracking N loose fields + forces independently.
99
+ - **A cleaner diff unit.** A change becomes *"this question's category changed"* or *"a category
100
+ parameter changed"* — one semantic delta — instead of a shower of add/remove field + add/edit
101
+ force/binding deltas. This is exactly the granularity the template diff-deploy work
102
+ (`.ai-assistance/projects/template-diff-deploy/`) wants.
103
+
104
+ ---
105
+
106
+ ## 5. The three decisions to nail in the meeting
107
+
108
+ **(a) Grow the forces effect grammar.** EPTIR's forces are modelled as a JSON-Logic `when` plus a
109
+ **closed effect set of `add_tag / remove_tag / set_value`**. That set does **not** include
110
+ `show/hide field|section` or `set_required` — which is precisely the flow Oscar is describing. The
111
+ category concept cannot expand into EPTIR until the effect grammar gains `show_field` / `hide_field`
112
+ / `show_section` / `hide_section` / `set_required` (all still closed, statically analysable). Decide
113
+ this and it flows into the parked EPTIR spec.
114
+
115
+ **(b) Which direction do we solve first — authoring or reverse-classification?** The same taxonomy
116
+ serves two uses:
117
+ - **Authoring** (design → categories → template): forward, and the write path is shipped
118
+ (`WorkflowAddForceInput`/`editForce`/`addBinding` via `executeWorkflowCommands`).
119
+ - **Reverse-classification** (read ~300 existing templates → cluster field+force clusters back into
120
+ categories): depends on **reading forces off a page**, which is the **one blocked path**
121
+ (see §6).
122
+
123
+ Recommend committing to **authoring-first**: it is unblocked, it directly serves the Sept 2026
124
+ CSV delivery, and it lets us validate the taxonomy before we depend on the blocked read path.
125
+
126
+ **(c) Closed catalog vs open user-defined archetypes.** A fixed catalog of ~N archetypes
127
+ (conditional-detail Select, mutually-exclusive Checklist reveal, computed risk matrix, …) vs. an
128
+ open, user-definable system. **Recommend closed-first** given the Sept 2026 deadline: a small closed
129
+ catalog is faster to build, trivial to conformance-test, and covers the common cases; leave an
130
+ escape hatch (raw fields+forces) for the long tail and revisit openness later.
131
+
132
+ ---
133
+
134
+ ## 6. The one concrete blocker
135
+
136
+ **Reading forces off a page is blocked.** `Query::PageWithForces` (used by
137
+ `ForceCompat#with_each_entry`) currently fails **client-side schema validation** (PageUnion
138
+ selections, `id` on binding types) — `10_forces_workflow_builder.md` lines 46-54. **Writing** forces
139
+ works today via the command bus. So authoring-first is fully unblocked; any reverse-classification of
140
+ existing templates is gated on fixing this read query. Name it in the room so ownership is clear.
141
+
142
+ ---
143
+
144
+ ## 7. Recommendation
145
+
146
+ 1. Adopt the **composite question archetype ("category")** as a new layer that compiles down into
147
+ the flat fields+forces IR (EPTIR) — do not fold it into the flat layer.
148
+ 2. **Extend EPTIR's forces effect grammar** with `show/hide field|section` + `set_required`
149
+ (decision 5a) — this is the concrete spec change that unblocks everything else.
150
+ 3. Ship a **closed catalog** of a handful of archetypes (5c), **authoring-first** (5b), targeting the
151
+ CSV pipeline for Sept 2026.
152
+ 4. File **fixing `Query::PageWithForces`** as the tracked prerequisite for reverse-classification
153
+ (§6), but do not let it block the authoring track.
154
+ ```
@@ -0,0 +1,75 @@
1
+ #!/usr/bin/env bash
2
+ # bridge-inbox-check.sh -- session-start actuator for the bridge inbox.
3
+ #
4
+ # Closes an actuation-coverage gap: unread cross-project bridge messages used to be
5
+ # visible only if someone manually ran a command. This is the trigger/clock that
6
+ # surfaces them automatically at session start, mirroring the standards-inbox report
7
+ # line (see CLAUDE.md and standards/governance/standards-request-protocol.md).
8
+ #
9
+ # DETERMINISTIC and zero-LLM: it just scans .ai-assistance/bridge/inbox/ for unread
10
+ # messages (STATUS: PENDING) and prints one summary line plus one line per message:
11
+ #
12
+ # [bridge] N unread message(s)
13
+ # - {title} from {FROM}
14
+ #
15
+ # Contract (mirrors the other hook scripts):
16
+ # - Silent (no output) and exit 0 when the inbox is empty or has no unread messages.
17
+ # - NEVER fails fatally -- always exits 0, even on a malformed file or missing dir.
18
+ # - No side effects: read-only, does not touch or move any message.
19
+ #
20
+ # Wired as a SessionStart hook in .claude/settings.json:
21
+ # bash scripts/bridge-inbox-check.sh 2>/dev/null || true
22
+ #
23
+ # Manual run (same output): bash scripts/bridge-inbox-check.sh
24
+ set -u
25
+
26
+ # Resolve the bridge inbox from this script's location so it works from any cwd.
27
+ # This script is deployed at .ai-assistance/scripts/, so the inbox is a sibling
28
+ # of the scripts dir: ../bridge/inbox (mirrors bridge-init.sh in this repo).
29
+ SCRIPT_DIR="$(cd "$(dirname "$0")" 2>/dev/null && pwd)" || exit 0
30
+ INBOX="$SCRIPT_DIR/../bridge/inbox"
31
+
32
+ # No inbox -> nothing to report.
33
+ [ -d "$INBOX" ] || exit 0
34
+
35
+ # An "unread" message is a *.md file whose header carries STATUS: PENDING.
36
+ # (PENDING = not yet picked up by the receiving agent; IN_PROGRESS/DONE/FAILED are read.)
37
+ unread=""
38
+ count=0
39
+
40
+ for f in "$INBOX"/*.md; do
41
+ # Guard against the literal glob when the folder is empty.
42
+ [ -e "$f" ] || continue
43
+
44
+ # Only inspect the header block (first ~15 lines) -- cheap and format-stable.
45
+ header="$(head -n 15 "$f" 2>/dev/null || true)"
46
+
47
+ # Unread == STATUS: PENDING (case-insensitive, tolerant of surrounding whitespace).
48
+ printf '%s\n' "$header" | grep -Eiq '^[[:space:]]*STATUS:[[:space:]]*PENDING[[:space:]]*$' || continue
49
+
50
+ # Title: the text after "# TASK:" on line 1; fall back to the filename.
51
+ title="$(printf '%s\n' "$header" \
52
+ | grep -Ei '^#[[:space:]]*TASK:' \
53
+ | head -n 1 \
54
+ | sed -E 's/^#[[:space:]]*TASK:[[:space:]]*//' 2>/dev/null || true)"
55
+ [ -n "$title" ] || title="$(basename "$f" .md)"
56
+
57
+ # Sender: the text after "FROM:"; fall back to "unknown".
58
+ from="$(printf '%s\n' "$header" \
59
+ | grep -Ei '^[[:space:]]*FROM:' \
60
+ | head -n 1 \
61
+ | sed -E 's/^[[:space:]]*FROM:[[:space:]]*//I' 2>/dev/null || true)"
62
+ [ -n "$from" ] || from="unknown"
63
+
64
+ count=$((count + 1))
65
+ unread="${unread}- ${title} from ${from}
66
+ "
67
+ done
68
+
69
+ # Silent when there is nothing unread.
70
+ [ "$count" -gt 0 ] || exit 0
71
+
72
+ printf '[bridge] %d unread message(s)\n' "$count"
73
+ printf '%s' "$unread"
74
+
75
+ exit 0
@@ -0,0 +1,181 @@
1
+ ---
2
+ name: project-self-docs
3
+ category: knowledge
4
+ version: 0.2.0
5
+ description: >
6
+ Generate and maintain a project's own committed self-documentation set (a stable-schema
7
+ OVERVIEW/ARCHITECTURE/CONVENTIONS/INTEGRATIONS/STATUS/COMPLIANCE/OPERATIONS) plus a machine-readable index and
8
+ changelog, so the project can later feed a central coordination hub. A thin orchestrator:
9
+ it reuses ai-discovery/code-specs/discovery-manifest findings and a deterministic scanner,
10
+ and never invents a documentation format. Invoke to create, refresh, or verify self-docs.
11
+ triggers:
12
+ - self docs
13
+ - self-documenting
14
+ - project-self-docs
15
+ - generate project docs
16
+ - refresh self docs
17
+ - self document this project
18
+ - hub docs
19
+ applicable_to:
20
+ - any
21
+ ---
22
+
23
+ # project-self-docs
24
+
25
+ ## Role
26
+
27
+ You maintain a project's SELF-DOCUMENTATION set: a small, stable set of committed markdown docs
28
+ that describe what the project is and how it is built, plus a machine-readable index and changelog
29
+ that a central coordination hub can consume. You are a THIN ORCHESTRATOR -- you compose facts that
30
+ `ai-discovery`, `code-specs`, and the deterministic scanner already produce; you do NOT re-analyse
31
+ the codebase from scratch, and you do NOT invent a new documentation format. Everything you write
32
+ is idempotent and re-runnable: running twice on an unchanged repo changes nothing.
33
+
34
+ This capability is the per-project half of the synergy hub (ADR-015, Section 8; and ADR-014). It exists
35
+ to detune the fleet by making reuse possible -- a project that documents itself against a shared
36
+ schema is a project others can find and build on instead of duplicating.
37
+
38
+ ---
39
+
40
+ ## On invocation
41
+
42
+ Before any other output, do both of the following:
43
+
44
+ 1. Print to the user:
45
+ `[skill: project-self-docs] <one-line description of what you are about to do>`
46
+ For example:
47
+ - `[skill: project-self-docs] scanning repo and reporting self-doc status`
48
+ - `[skill: project-self-docs] refreshing STATUS.md and ARCHITECTURE.md from current state`
49
+ - `[skill: project-self-docs] creating the self-doc set for the first time`
50
+
51
+ 2. Append a usage record to `.ai-assistance/local/kpi/usage-<YYYY-WNN>.jsonl`
52
+ (ISO week format: `YYYY-WNN`, e.g. `2026-W24`):
53
+ ```json
54
+ {"component": "skill/project-self-docs", "action": "invoked", "detail": "<same one-liner>", "ts": "<ISO timestamp>", "session_id": "<if known from token-budget session>"}
55
+ ```
56
+ Use `action` values: `invoked`, `completed`, `skipped`. If the file or directory does not
57
+ exist, create it. If writing fails, continue silently.
58
+
59
+ ---
60
+
61
+ ## Before answering -- run the deterministic scan FIRST
62
+
63
+ The deterministic scanner does the zero-LLM half. ALWAYS run it before writing any prose, so you
64
+ spend interactive tokens only on the semantic gaps it cannot fill:
65
+
66
+ ```bash
67
+ python .ai-assistance/skills/project-self-docs/scripts/self_docs_scan.py --json
68
+ ```
69
+
70
+ (From ep-ai-standards itself, the script lives at
71
+ `skills-library/project-self-docs/scripts/self_docs_scan.py`.)
72
+
73
+ The observation tells you deterministically: repo type, real repo name (from the remote), git
74
+ head/branch, top-level structure, key config + CI files, activity clusters (hot paths), which
75
+ existing inputs you can REUSE (`ai-discovery` `repo-context.md`/`conventions.md`, `code-specs`
76
+ under `.ai-assistance/code/`), and which self-docs already exist with their current content hash.
77
+
78
+ **Reuse before you write.** If `repo-context.md`, `conventions.md`, or code-specs exist, compose
79
+ the self-docs from them -- do not re-derive the same facts. If they do NOT exist, propose running
80
+ `ai-discovery` (first-time) or `code-specs` (per-area) first, rather than duplicating their work
81
+ here.
82
+
83
+ ---
84
+
85
+ ## The self-doc set (stable schema -- do not change filenames)
86
+
87
+ Copy the templates from `templates/project-self-docs/` into the project's self-docs directory
88
+ (default `docs/self-docs/`; configurable). The set is intentionally small and fixed:
89
+
90
+ | File | Covers |
91
+ |---|---|
92
+ | `OVERVIEW.md` | What the project is, who it serves, current priority (one screen) |
93
+ | `ARCHITECTURE.md` | Top-level structure, key components, how they fit, entry points |
94
+ | `CONVENTIONS.md` | Coding style, branch naming, commit style, test conventions |
95
+ | `INTEGRATIONS.md` | External services + data + permissions, upstream/downstream repos, dependencies |
96
+ | `STATUS.md` | Active work, migration state, known risks, open questions |
97
+ | `COMPLIANCE.md` | ISO-27001/audit, data classes + PII, third-party vendors, AI content, leak controls |
98
+ | `OPERATIONS.md` | Infrastructure, public exposure + Cloudflare, DevOps audit runbook, usage KPIs/ROI |
99
+
100
+ The set is the ROVO audit/compliance/discovery surface once mirrored into the EP Projects Register
101
+ (`docs/strategy/self-docs-audit-schema.md`). Schema is `1.1`. Each file carries YAML frontmatter
102
+ (`schema_version`, `repo`, `doc`, `last_generated`,
103
+ `source_head`, `review_status`) so the hub can parse it and so a human review gate is explicit.
104
+ Do NOT add, rename, or drop files without bumping the template `schema_version` and the scanner's
105
+ `SELF_DOC_SET` together (they are checked as a pair).
106
+
107
+ These docs are COMMITTED (not gitignored) -- they are the shareable artefact, unlike the
108
+ gitignored `.ai-assistance/local/` context that `ai-discovery` writes.
109
+
110
+ ---
111
+
112
+ ## Workflow
113
+
114
+ 1. **Scan.** Run `self_docs_scan.py --json`. Read the observation.
115
+ 2. **Reuse.** Pull facts from the existing inputs it lists. Only genuinely-missing semantic content
116
+ is yours to write.
117
+ 3. **Fill / refresh.** For each self-doc: if missing, copy the template and fill it; if present,
118
+ update only the sections whose underlying facts changed (structure, hot paths, status). Keep
119
+ `review_status: draft` on anything a human has not confirmed.
120
+ 4. **Batch heavy semantic work to Gemini.** For a large or unfamiliar repo, draft
121
+ ARCHITECTURE/OVERVIEW via the `gemini-assist` skill (no-training tier) rather than on
122
+ interactive Claude tokens -- per the token-frugality rule. Verify Gemini's output against the
123
+ scanner facts before writing.
124
+ 5. **Write the index + changelog (deterministic).** Run:
125
+ ```bash
126
+ python .ai-assistance/skills/project-self-docs/scripts/self_docs_scan.py --write
127
+ ```
128
+ This rewrites `docs/self-docs/self-docs-index.json` (the machine surface the hub reads) and
129
+ APPENDS one line per changed doc to `docs/self-docs/CHANGES.jsonl`. A no-op re-run appends
130
+ nothing -- this is what makes the capability idempotent.
131
+ 6. **Human review before any egress.** Leave publishing to a human-reviewed step (see below).
132
+
133
+ ---
134
+
135
+ ## Publish / egress -- CONFIGURABLE, default local-only (OPEN DECISION)
136
+
137
+ The Confluence publish destination is an OPEN DECISION being made separately (existing EPAI space
138
+ vs a new dedicated hub space vs the private EP-AI-MGMT space). Therefore:
139
+
140
+ - **Default: write to the repo/local filesystem ONLY.** This skill does not push to Confluence.
141
+ - The publish target is a PARAMETER, not a hardcoded value. When the decision is made, the egress
142
+ step becomes a call to the existing seed pipeline (`scripts/seed-epai-project-docs.py` pattern)
143
+ pointed at the chosen space, gated by human review.
144
+ - <!-- TODO(hub-target): wire egress to the chosen Confluence space once the publish-target
145
+ decision lands. Until then, self-docs live in docs/self-docs/ and are consumed from git. -->
146
+
147
+ Never emit `r` (a result) straight to a shared surface: a human review gate sits between the
148
+ generated self-docs and any hub publish. This is a damping control -- keep it.
149
+
150
+ ---
151
+
152
+ ## Actuation
153
+
154
+ - **Actor:** a developer (or `ai-discovery`/`project-cycle` calling this skill) IN THE REPO THAT
155
+ DOCUMENTS ITSELF, never an unattended agent -- and NEVER an ep-ai-standards session reaching into
156
+ another repo to generate that repo's self-docs. ep-ai-standards AUTHORS + DEPLOYS this skill and
157
+ AGGREGATES/publishes results; each repo RUNS it in its own session. (Running it on ep-ai-standards
158
+ ITSELF is fine -- that is why `docs/self-docs/` exists here.)
159
+ - **Trigger / clock:** on demand; recommended at project cycle-end (`project-cycle`) and after a
160
+ structural change. There is no autonomous fleet-wide trigger yet -- that depends on the unbuilt
161
+ ADR-014 upkeep loop; note this rather than inventing an actuator.
162
+ - **Coverage on neglect:** if self-docs are never refreshed, the scanner's `content_hash` +
163
+ `source_head` in the index let the hub (and `project-review`) flag them as stale. The doc is
164
+ never silently trusted; staleness is detectable.
165
+ - **Write authority:** writes ONLY under the self-docs directory (default `docs/self-docs/`), plus
166
+ the KPI usage line. It does not touch source code, `.ai-assistance/local/`, git remotes, or any
167
+ network target.
168
+
169
+ ---
170
+
171
+ ## What you do not do
172
+
173
+ - Do not invent a documentation format -- use the fixed self-doc set and templates.
174
+ - Do not re-analyse the codebase from scratch -- reuse `ai-discovery`, `code-specs`, and the
175
+ scanner; propose running those skills if their inputs are missing.
176
+ - Do not publish to Confluence or any network target -- egress is a separate, human-gated,
177
+ as-yet-undecided step.
178
+ - Do not write outside `docs/self-docs/` (or the configured self-docs dir).
179
+ - Do not run heavy semantic drafting on interactive Claude tokens for large repos -- batch to
180
+ Gemini (no-training tier).
181
+ - Do not mark a doc `review_status: reviewed` -- only a human does that.