@sentientui/mcp 0.18.3 → 0.19.0

package/AGENTS.md

@@ -33,14 +33,14 @@ Server keys start with `sk_` — required for the management API. Public keys (`
 | `create_project` | Onboard a new user: creates a project + returns its `pk_` key. Needs an account login (OAuth) — refuses `sk_` project keys and demo tokens. Follow with `get_integration_guide` |
 | `get_project_stats` | Health status + 24h event/session volume |
 | `list_components` | Components + variant counts + impression totals |
-| `get_variant_performance` | CVR current vs prior window + momentum (range/from/to params; default 7d) |
+| `get_variant_performance` | Per-(component, variant) rate with its n (converting / exposed sessions, range/from/to; default 7d), plus the server's evidence verdict on that windowed rate vs the component's baseline (`verdict`: ahead / behind / unclear; `verdictBasis` says window, or all_time on older APIs) and the all-time verdict as reliability (`evidenceState`, `reliabilityVerdict`). Only ahead/behind is a result; the preceding-window change is an untested descriptor |
 | `get_persona_breakdown` | Cluster distribution + avg reliability per cluster |
-| `get_goal_funnel` | Goal hit counts + per-variant completion rates |
+| `get_goal_funnel` | Goal hit counts + per-variant all-time completion rates, each with its n and a low-sample flag (descriptive — no verdict) |
 | `list_guardrail_events` | Auto-paused variants (last 24h) |
 | `get_layout_stats` | Per-persona layout order + bandit pulls/reward |
-| `get_insights` | Narrator bullets + (Growth) advisor bullets |
+| `get_insights` | Ranked measured findings (tier + sample, low-sample flagged; `limit`, default 10) + (Growth) AI narrations, which are unmeasured |
 | `refresh_insights` | Async trigger — wait ~15s then call `get_insights` |
-| `get_variant_brief` | **Start here to optimize a component.** Insight-driven brief for writing a new CODE-NATIVE variant: performance, audience, insights, data-sufficiency + best-practice fallback, and code instructions |
+| `get_variant_brief` | **Start here to optimize a component.** Evidence-driven brief for writing a new CODE-NATIVE variant: each arm's rate with n and server evidence verdict, audience, measured findings for the component, evidence state (EMPTY / COLLECTING / DIRECTIONAL / DECIDED) + best-practice fallback, and code instructions |
 | `create_variant` | Creates a no-code **managed text** DRAFT — user must activate in dashboard. Fallback only; not for code-native variants |
 | `pause_variant` | Stops traffic; no MCP resume |
 | `get_cell_matrix` | "Who sees what": per-(visitor type, region) generation state + traffic share. A missing cell = they see the original |
@@ -52,18 +52,18 @@ Server keys start with `sk_` — required for the management API. Public keys (`
 **Diagnose a conversion drop**
 1. `get_project_stats` — confirm events are flowing
 2. `get_goal_funnel` — find which goals/variants are down
-3. `get_variant_performance` — check CVR momentum
+3. `get_variant_performance` — check each arm's rate with its n and the server's evidence verdict
 4. `list_guardrail_events` — see if the guardrail fired
 5. `refresh_insights` → wait 15s → `get_insights` — read AI summary
 
 **Optimize a component with a new variant (code-native — recommended)**
 1. `list_components` — confirm the component ID
-2. `get_variant_brief` — pull performance, audience, insights, data-sufficiency, best-practice priors, and code instructions for that component
+2. `get_variant_brief` — pull per-arm evidence, audience, measured findings, evidence state, best-practice priors, and code instructions for that component
 3. Find `<Adaptive id="...">` in the repo and add a new on-brand variant key to its `variants` map (and to the `AdaptiveRoot` `components` list if SSR). Match the project's existing components and design system.
 4. Do **not** call `create_variant`. Commit, push, deploy — the variant auto-registers on the first assignment and goes live.
 5. Optional: enable shadow mode (project-level, in dashboard settings — it applies to the whole project, not a single component) first to validate before serving.
 
-Data sufficiency (from the brief) drives the change: when it reports **SUFFICIENT**, target the specific weakness; when **COLLECTING** or **EMPTY** (no data yet), apply the best-practice priors for the project's context type — use your best judgment and make one conservative change.
+The evidence state (from the brief) drives the change. It is keyed on the server's evidence tiers, never on a traffic count: only **DECIDED** (an arm reliably ahead of or behind the baseline, at the multiple-comparison-corrected strong bar) is a result to build on. **DIRECTIONAL** (early/moderate/inconclusive signals), **COLLECTING** (below the 100-visit floor per arm) and **EMPTY** all mean: apply the best-practice priors for the project's context type and make one conservative change. Never call an arm better or worse from its raw rate.
 
 **Add a no-code managed text variant (fallback — text only, less powerful)**
 1. `list_components` — confirm the component ID
@@ -85,7 +85,8 @@ Data sufficiency (from the brief) drives the change: when it reports **SUFFICIEN
 - Presenting stale insights without checking `isStale` — always check, and offer to `refresh_insights`
 - Assuming `create_variant` is live — it's always a draft
 - Calling `create_variant` for a variant that's being written in code — it creates an empty managed draft, not your code variant. Code-native variants auto-register on deploy; only use `create_variant` for no-code/managed variants whose content lives in SentientUI
-- Optimizing a component without calling `get_variant_brief` first — it gives you the performance, audience, insights, and data-sufficiency fallback you need to write a variant that actually makes sense (and the right steps when there is no data yet)
+- Optimizing a component without calling `get_variant_brief` first — it gives you the per-arm evidence, audience, findings, and evidence-state fallback you need to write a variant that actually makes sense (and the right steps when there is no data yet)
+- Ranking variants by raw conversion rate — a rate on 10 sessions is noise. Only a server `verdict` of ahead/behind is a comparison; every rate carries its n for this reason
 - Calling `pause_variant` without warning the user it's irreversible via MCP
 - Using a `pk_` key instead of `sk_` — it will be rejected
 - Interpreting `avgReward` in layout stats as a conversion rate — it's a bandit reward signal, not CVR

package/README.md

@@ -88,17 +88,17 @@ A sandboxed demo token is provisioned automatically — 10 calls/month, read-onl
 | `create_project` | Create a new project and return its `pk_` public key (account login required — not usable with an `sk_` project key or demo token) |
 | `get_project_stats` | Events, sessions, agent calls, and health status |
 | `list_components` | All adaptive components with variant counts |
-| `get_variant_performance` | CVR, momentum, and impressions per variant (windowed: range 7d/30d/90d/all or from/to; default 7d vs prior 7d) |
-| `get_insights` | AI-generated narrator observations and advisor recommendations |
+| `get_variant_performance` | Per-(component, variant) conversion with its sample, plus the server's evidence verdict vs the baseline (windowed rates: range 7d/30d/90d/all or from/to; default 7d; the verdict judges the windowed rate, with all-time reliability reported separately — all-time only on APIs without window evidence) |
+| `get_insights` | Ranked measured findings with tier and sample (low samples flagged), plus AI narrations marked as unmeasured |
 | `refresh_insights` | Trigger fresh AI insight generation |
 | `get_persona_breakdown` | Visitor cluster distribution with reliability scores |
-| `get_goal_funnel` | Goal hit counts and conversion rates per variant (windowed; default 30d) |
+| `get_goal_funnel` | Goal hit counts and conversion rates (windowed; default 30d), plus all-time per-variant rates with their n |
 | `list_goals` | Defined goals (id, role, event, status) — includes goals with no conversions yet, so agents can wire a dashboard-defined goal into code |
 | `list_guardrail_events` | Variants auto-paused by the guardrail (last 24h) |
 | `get_layout_stats` | Per-persona section layout rankings and reward weights |
 | `get_integration_guide` | SentientUI adaptive-ladder setup guide (static — same for every project) |
-| `get_test_brief` | What to test and how, for the project's current state |
-| `get_variant_brief` | Insight-driven brief for writing a new **code-native** variant: performance, audience, insights, data-sufficiency (with a best-practice fallback when there's no data yet), and step-by-step code instructions |
+| `get_test_brief` | Ready-to-paste test code (RTL, mock server, Playwright/Cypress) that forces one of the component's real variants |
+| `get_variant_brief` | Evidence-driven brief for writing a new **code-native** variant: per-arm rates with n and evidence verdicts, audience, measured findings, an evidence state (with a best-practice fallback while nothing is decided), and step-by-step code instructions |
 | `create_variant` | Create a no-code (managed) text variant (Starter+) — fallback for text-only variants without a code change |
 | `pause_variant` | Pause a variant to stop traffic assignment |
 | `get_cell_matrix` | The "Who sees what" matrix: which visitor types have an AI-generated version live in each personalizable region, with traffic share and auto-fill status |
@@ -109,7 +109,7 @@ A sandboxed demo token is provisioned automatically — 10 calls/month, read-onl
 >
 > **Code-native vs no-code variants.** Variants you declare in your app (`<Adaptive variants={{…}}>`) register automatically the first time the SDK requests an assignment after you deploy — they go live immediately and need **no** `create_variant` call. Use `create_variant` only for **no-code** variants whose content is stored in SentientUI and rendered without a code change; these start as drafts you activate from the dashboard.
 >
-> **Optimizing a component?** Call `get_variant_brief` first. It returns the component's performance, audience, insights, and a data-sufficiency assessment — then your AI assistant writes a new on-brand variant directly into your code (which auto-registers on deploy). When there's no data yet, the brief falls back to best-practice priors for your project's context type so the assistant still makes a sensible change.
+> **Optimizing a component?** Call `get_variant_brief` first. It returns each variant's rate with its sample and the server's evidence verdict, the audience, measured findings, and an evidence-state assessment — then your AI assistant writes a new on-brand variant directly into your code (which auto-registers on deploy). While nothing is decided, the brief falls back to best-practice priors for your project's context type so the assistant still makes a sensible change.
 
 ## Example prompts