@nurix/etna 0.18.0 → 0.18.2

package/data/harnesses/discover/rules/evaluation.md

@@ -1,97 +1,78 @@
 # Evaluation Templates
 
-`evaluation` reads every `dimensions/<dim>/_summary.md` (its **provenance frontmatter** +
-Findings) and reconciles them into **cross-dimension** documents under
-`research/<target>/evaluation/`. The value is the reconciliation — cross-checking views into one
-coherent picture, **weighted by how each claim was obtained**, with disagreements flagged.
+`evaluation` reads every `dimensions/<dim>/_summary.md` (its **provenance frontmatter** + Findings) and reconciles them into **cross-dimension** documents under `research/<target>/evaluation/`. The value is the reconciliation — cross-checking views into one coherent picture, **weighted by how each claim was obtained**, with disagreements flagged.
+
+> This file governs the **four reconciliation rollups** (technology-architecture, product-features, data-model-api-surface, competitive-positioning) + the README. The three **experiential** docs that also live under `evaluation/` — `information-architecture.md`, `ux-flows.md`, `feature-coverage.md` — are produced by **Mode 4 (Cartography)** and governed by `cartography.md`; Evaluation's rollups are their inputs (the entity spine → IA surface cards; the feature map → the coverage matrix), and Cartography's coverage scorecard feeds Mode 5's experiential-coverage axis.
 
 **Three rules for every evaluation doc:**
-1. **Anchor + tag every claim** with its dimension *and provenance*: `(api: raw/endpoint-catalog.md
-   · openapi-verbatim · high)`. The method + confidence come verbatim from the source dimension's
-   `_summary.md` frontmatter.
-2. **Weight, don't average.** A claim's confidence = the **max** of its sources' confidence,
-   **promoted one band only if corroborated by ≥2 *independent* dimensions**, capped at source
-   confidence when single-source. Conflicts are **flagged, never averaged**. **Tiebreaker — direct
-   observation outranks inference.** When two sources conflict on the *same* fact, a directly-observed
-   value beats an inferred one regardless of dimension or stated confidence (a session-observed backend
-   engine beats a DNS-inferred one; a live HTTP probe beats a single NXDOMAIN result). The inferred
-   side is rendered tentative and never used to override the observation or to issue cross-dimension directives.
-3. **Cross-check, don't concatenate.** Agree → state as fact. Disagree / single-source / low-conf →
-   render hedged ("observed but unverified…") and flag in the doc's reconciliation section.
-
-**Independence (for the promotion rule):** `codebase`, `api`, `session`, `wire-capture`,
-`distribution-artifacts` are mutually independent and independent of marketing. **`website` and
-`docs` are NOT independent** for *feature claims* (docs often restate marketing) — two of them
-agreeing is still one source. A bundle-declared route + a session-observed route ARE independent.
-**Same artifact ⇒ one source.** Two dimensions that string-mine the *same* static asset (one bundle /
-one spec / one binary) are **one source** — never a band-bump — even when they report different
-counts; the divergent counts are a **reported range, not a flagged conflict** (`deployed-client-bundle`
-278 vs `website` 361 over one bundle is one finding with a range, not corroboration and not a conflict).
-**OSS distribution clients are an independent corroboration lane.** An API-surface claim resting on a
-single *third-party* wrapper (a community MCP server, an unofficial SDK) is tentative — but a
-**first-party open-source integration client** (`distribution-artifacts`: a Unity/Figma/CLI plugin or
-published SDK source) that calls the same routes is **independent of both marketing and the third-party
-wrapper**, and corroborates it → promotes. (A legacy/granular route family seen only in a third-party
-MCP becomes a real finding once the official OSS client calls it too.)
-
-**Open-core lineage as an absence oracle (independent corroboration of the open/closed seam).** When a
-target is an **open-core product with a closed cloud edition** (a clonable `codebase` lineage repo + a
-closed `session` runtime), a feature **verified ABSENT in the lineage repo** (grep/structure-map) **and
-observed PRESENT in the cloud session** is **two independent lanes agreeing on the *seam*** → promotes the
-open/closed-boundary claim to fact (the cloud-feature value itself stays a direct `session` observation).
-The lineage repo is as useful as an *absence map* (what is NOT in the open floor) as it is a presence map;
-"verified-absent-in-CE + observed-in-cloud" is independent of marketing and of the session alone. (Origin:
-Akeneo — Reference Entities / DAM / Rules / Workflows observed in Serenity + grep-absent in Community Edition.)
-
-**Closed-feedback confidence cap.** When the `community` dimension finds **no first-party feedback
-loop** (no public issues / roadmap / changelog / forum — feedback funnelled to a gated channel), its
-user-pain / most-requested-feature claims are **inference-from-changelog only → capped at
-`low`/tentative**, never promoted on changelog frequency alone. The surface that *would* corroborate
-user sentiment is `external-reputation` (a deferred dimension) — flag it as recommended-for-this-run
-when this cap fires (see discovery Part C deferred-dimension trigger).
-
-**Write-side-unobserved cap.** A `session` dimension carrying `write_side_observed: false` (a
-Pass-1-only run) has observed feature *existence/gating* but **not** feature *execution behavior*. Any
-marketed generate/submit/launch *behavior* claim therefore stays **tentative and is never promoted to
-fact** from marketing/docs alone — render the write-side gap **once** from the frontmatter flag, not
-re-hedged per rollup.
-
-**Presentation mapping:** high → stated as fact; low/single-source → hedged as tentative;
-conflicted → flagged. (This is the "lighter" form of the provenance-weighting — done inline in the
-prose, not via a separate `claims.yaml`.)
+
+1. **Anchor + tag every claim** with its dimension _and provenance_: `(api: raw/endpoint-catalog.md · openapi-verbatim · high)`. The method + confidence come verbatim from the source dimension's `_summary.md` frontmatter.
+2. **Weight, don't average.** A claim's confidence = the **max** of its sources' confidence, **promoted one band only if corroborated by ≥2 _independent_ dimensions**, capped at source confidence when single-source. Conflicts are **flagged, never averaged**. **Tiebreaker — direct observation outranks inference.** When two sources conflict on the _same_ fact, a directly-observed value beats an inferred one regardless of dimension or stated confidence (a session-observed backend engine beats a DNS-inferred one; a live HTTP probe beats a single NXDOMAIN result). The inferred side is rendered tentative and never used to override the observation or to issue cross-dimension directives.
+3. **Cross-check, don't concatenate.** Agree → state as fact. Disagree / single-source / low-conf → render hedged ("observed but unverified…") and flag in the doc's reconciliation section.
+
+**Independence (for the promotion rule):** `codebase`, `api`, `session`, `wire-capture`, `distribution-artifacts` are mutually independent and independent of marketing. **`website` and `docs` are NOT independent** for _feature claims_ (docs often restate marketing) — two of them agreeing is still one source. A bundle-declared route + a session-observed route ARE independent. **Same artifact ⇒ one source.** Two dimensions that string-mine the _same_ static asset (one bundle / one spec / one binary) are **one source** — never a band-bump — even when they report different counts; the divergent counts are a **reported range, not a flagged conflict** (`deployed-client-bundle` 278 vs `website` 361 over one bundle is one finding with a range, not corroboration and not a conflict). **OSS distribution clients are an independent corroboration lane.** An API-surface claim resting on a single _third-party_ wrapper (a community MCP server, an unofficial SDK) is tentative — but a **first-party open-source integration client** (`distribution-artifacts`: a Unity/Figma/CLI plugin or published SDK source) that calls the same routes is **independent of both marketing and the third-party wrapper**, and corroborates it → promotes. (A legacy/granular route family seen only in a third-party MCP becomes a real finding once the official OSS client calls it too.)
+
+**Open-core lineage as an absence oracle (independent corroboration of the open/closed seam).** When a target is an **open-core product with a closed cloud edition** (a clonable `codebase` lineage repo + a closed `session` runtime), a feature **verified ABSENT in the lineage repo** (grep/structure-map) **and observed PRESENT in the cloud session** is **two independent lanes agreeing on the _seam_** → promotes the open/closed-boundary claim to fact (the cloud-feature value itself stays a direct `session` observation). The lineage repo is as useful as an _absence map_ (what is NOT in the open floor) as it is a presence map; "verified-absent-in-CE + observed-in-cloud" is independent of marketing and of the session alone. (Origin: Akeneo — Reference Entities / DAM / Rules / Workflows observed in Serenity + grep-absent in Community Edition.)
+
+**Closed-feedback confidence cap.** When the `community` dimension finds **no first-party feedback loop** (no public issues / roadmap / changelog / forum — feedback funnelled to a gated channel), its user-pain / most-requested-feature claims are **inference-from-changelog only → capped at `low`/tentative**, never promoted on changelog frequency alone. The surface that _would_ corroborate user sentiment is `external-reputation` (a deferred dimension) — flag it as recommended-for-this-run when this cap fires (see discovery Part C deferred-dimension trigger).
+
+**Write-side-unobserved cap.** A `session` dimension carrying `write_side_observed: false` (a Pass-1-only run) has observed feature _existence/gating_ but **not** feature _execution behavior_. Any marketed generate/submit/launch _behavior_ claim therefore stays **tentative and is never promoted to fact** from marketing/docs alone — render the write-side gap **once** from the frontmatter flag, not re-hedged per rollup.
+
+**Presentation mapping:** high → stated as fact; low/single-source → hedged as tentative; conflicted → flagged. (This is the "lighter" form of the provenance-weighting — done inline in the prose, not via a separate `claims.yaml`.)
 
 ---
 
 ## 1. `evaluation/technology-architecture.md`
+
 ```markdown
 # <Target> — Technology & Architecture
-## TL;DR                  (3–5 sentences; the single most distinctive choice)
-## Stack                  (languages/frameworks/build/runtime — table, each anchored+tagged)
-## Architecture           (client/server split, packages+deps, runtime/sync/data model, transport)
-## How it's built         (monorepo layout, public/private, codegen, CI, release cadence)
-## Notable design choices  (the 3–5 worth stealing + why)
-## Cross-dimension reconciliation  (agreements/disagreements + confidence per claim)
+
+## TL;DR (3–5 sentences; the single most distinctive choice)
+
+## Stack (languages/frameworks/build/runtime — table, each anchored+tagged)
+
+## Architecture (client/server split, packages+deps, runtime/sync/data model, transport)
+
+## How it's built (monorepo layout, public/private, codegen, CI, release cadence)
+
+## Notable design choices (the 3–5 worth stealing + why)
+
+## Cross-dimension reconciliation (agreements/disagreements + confidence per claim)
+
 ## Open questions
 ```
 
 ## 2. `evaluation/product-features.md`
+
 ```markdown
 # <Target> — Product Features & Capabilities
-## What it does           (one paragraph)
-## Feature map            (every capability · user · maturity GA/beta/docs-only — anchored+tagged)
-## User-facing surfaces   (dashboard/editor/components/CLI/widgets)
-## User journeys          (the 2–3 primary flows end-to-end)
-## Gaps & rough edges     (from community signal)
-## Cross-dimension reconciliation  (marketing claims vs. what code/docs actually support — flag over-claims)
+
+## What it does (one paragraph)
+
+## Feature map (every capability · user · maturity GA/beta/docs-only — anchored+tagged)
+
+## User-facing surfaces (dashboard/editor/components/CLI/widgets)
+
+## User journeys (the 2–3 primary flows end-to-end)
+
+## Gaps & rough edges (from community signal)
+
+## Cross-dimension reconciliation (marketing claims vs. what code/docs actually support — flag over-claims)
 ```
 
 ## 3. `evaluation/data-model-api-surface.md`
+
 ```markdown
 # <Target> — Data Model & API / SDK Surface
-## Core entities          (domain objects + relations; ER-style — when the runtime exposes an RBAC verb
+
+## Core entities (domain objects + relations; ER-style — when the runtime exposes an RBAC verb
+
                            taxonomy, derive the entity set from it: `X_CREATE/VIEW/EDIT/DELETE` ⇒ entity X)
-## Integration surface    (SDK/packages public API · REST/HTTP catalog · realtime/wire protocol)
-## Published-vs-app-own API diff  (emit WHENEVER the target publishes a developer/partner/MCP API —
+
+## Integration surface (SDK/packages public API · REST/HTTP catalog · realtime/wire protocol)
+
+## Published-vs-app-own API diff (emit WHENEVER the target publishes a developer/partner/MCP API —
+
                             common for mature SaaS, but not universal. The app the SPA runs on is frequently
                             a different, broader, differently-authed API than the published one: list ops the
                             app uses that the public API omits, AND public ops the app never calls — both are
@@ -100,55 +81,68 @@ prose, not via a separate `claims.yaml`.)
                             REQUIRES paired inputs (ingestion §6): the pre-nav in-page tap [session app-own
                             paths] + the unauth bundle/route-manifest [published paths]. Missing a lane → the
                             diff is un-feedable; say which lane is absent rather than implying overlap.)
-## Auth & tenancy model   (token type, where it lives, scope expression)
-## API-path diff (adaptive, N-way over the runtime sources present)  (the seam rule: bundle-declared vs
+
+## Auth & tenancy model (token type, where it lives, scope expression)
+
+## API-path diff (adaptive, N-way over the runtime sources present) (the seam rule: bundle-declared vs
+
                             session-observed vs wire-observed. Title it for the sources ACTUALLY run —
                             enumerate only those lanes; an absent lane [session blocked / wire folded] is a
                             recorded gap, NOT a structural hole to apologize for. Dormant/unshipped routes
                             are findings.)
-## Cross-dimension reconciliation  (published surface [api/packages/docs] vs observed wire [session/
+
+## Cross-dimension reconciliation (published surface [api/packages/docs] vs observed wire [session/
+
                             wire-capture]; document drift — published=fact, observed-drift=tentative)
+
 ## Open questions
 ```
 
 ## 4. `evaluation/competitive-positioning.md`
+
 ```markdown
 # <Target> — Competitive Positioning
-## Positioning            (their pitch + market — website/blog)
-## Pricing & packaging    (plans, what gates what, unit economics — website pricing + platform/limits)
-## Differentiation        (claimed moat vs the teardown's read of the real moat — often infra/data)
-## Alternatives & comparison  (named competitors + how it differs)
-## Strategic read         (direction from changelog/blog)
+
+## Positioning (their pitch + market — website/blog)
+
+## Pricing & packaging (plans, what gates what, unit economics — website pricing + platform/limits)
+
+## Differentiation (claimed moat vs the teardown's read of the real moat — often infra/data)
+
+## Alternatives & comparison (named competitors + how it differs)
+
+## Strategic read (direction from changelog/blog)
 ```
 
 ---
 
 ## 5. `research/<target>/README.md` — the run story
+
 ```markdown
 # <Target> — Recon Run
+
 > One paragraph: what it is + why we ran recon.
-## Access grade           (the resolved vector — source/runtime/auth/presence — + matched case[s])
-## Dimensions covered     (table: dimension | status | method | confidence | headline finding)
-## Headline findings      (5–10, each anchored+tagged)
-## How to navigate         (00-scope-verdict.md, 00-recon-plan.md, dimensions/, evaluation/, 04-self-correction-proposal.md)
-## Lessons                 (candidate methodology learnings → feed self-correction)
+
+## Access grade (the resolved vector — source/runtime/auth/presence — + matched case[s])
+
+## Dimensions covered (table: dimension | status | method | confidence | headline finding)
+
+## Headline findings (5–10, each anchored+tagged)
+
+## How to navigate (00-scope-verdict.md, 00-recon-plan.md, dimensions/, evaluation/ [4 reconciliation rollups + Cartography's information-architecture / ux-flows / feature-coverage], 05-self-correction-proposal.md)
+
+## Lessons (candidate methodology learnings → feed self-correction)
+
 ## Reproduction
 ```
 
 ---
 
 ## Evaluation method
-1. Read every `dimensions/*/_summary.md` frontmatter first (the provenance index) — note method +
-   confidence + status + gaps per dimension.
+
+1. Read every `dimensions/*/_summary.md` frontmatter first (the provenance index) — note method + confidence + status + gaps per dimension.
 2. Read Findings; drop into specific `raw/` artifacts only when a claim needs a precise value.
-3. Draft the four rollups + the README — each rollup may be drafted by its own sub-agent dispatched
-   from the **main session** (overlapping reads, independent writes), then a single reconciliation
-   pass keeps them consistent.
-4. Tag every claim with provenance; weight per the rules above; flag every disagreement; record
-   every gap. A confident doc built on one low-confidence dimension is worse than an honest
-   "single-source, unverified."
-
-> **Classifying a raw observation** — to turn a storage-key prefix, an auth tell, an envelope shape,
-> or a realtime frame into a *named* stack finding, use the fingerprint tables in
-> `tradecraft.md` (storage→stack, auth-provider, API-envelope, realtime-trap). They name what
-> a finding *is*; this file governs how confidently to *state* it.
+3. Draft the four rollups + the README — each rollup may be drafted by its own sub-agent dispatched from the **main session** (overlapping reads, independent writes), then a single reconciliation pass keeps them consistent.
+4. Tag every claim with provenance; weight per the rules above; flag every disagreement; record every gap. A confident doc built on one low-confidence dimension is worse than an honest "single-source, unverified."
+
+> **Classifying a raw observation** — to turn a storage-key prefix, an auth tell, an envelope shape, or a realtime frame into a _named_ stack finding, use the fingerprint tables in `tradecraft.md` (storage→stack, auth-provider, API-envelope, realtime-trap). They name what a finding _is_; this file governs how confidently to _state_ it.