@stackwright-pro/otters 1.0.0-alpha.72 → 1.0.0-alpha.73

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@stackwright-pro/otters",
-  "version": "1.0.0-alpha.72",
+  "version": "1.0.0-alpha.73",
   "description": "Stackwright Pro Otter Raft - AI agents for enterprise features (CAC auth, API dashboards, government use cases)",
   "license": "SEE LICENSE IN LICENSE",
   "repository": {
@@ -25,7 +25,7 @@
     "access": "public"
   },
   "peerDependencies": {
-    "@stackwright-pro/mcp": "^0.2.0-alpha.106"
+    "@stackwright-pro/mcp": "^0.2.0-alpha.107"
   },
   "scripts": {
     "generate-checksums": "node scripts/generate-checksums.js",

package/src/checksums.json

@@ -5,7 +5,7 @@
     "stackwright-pro-api-otter.json": "18112d603646457cecdfd57851109ff9f9ff8f402646e0d54ddef88cf5547d91",
     "stackwright-pro-auth-otter.json": "1a21d9f23e250f23291124307e5a2a32af5a716566319a3290372dae3d02e637",
     "stackwright-pro-dashboard-otter.json": "4a9ac39e929866ae3c929aa5bcbba3d9483a1c251c35f26f267f3e7433506f0c",
-    "stackwright-pro-data-otter.json": "1ad3ed99bbe7b550f654c679a8c0ea3363b2c52031042cd177c6e5f9e1c50a21",
+    "stackwright-pro-data-otter.json": "be4e3d6f530ae2d6b79d6412e8d2d6defb3caffddedf5be4f928976a72206074",
     "stackwright-pro-designer-otter.json": "69a6c09fbb54f971c48eae7fd52faa63cf79be2923e435aca9ff802e8d541d0f",
     "stackwright-pro-domain-expert-otter.json": "14d77cade020f44d1b5a2b406e2599501f3466ee90ca4248500a441d419bc59c",
     "stackwright-pro-foreman-otter.json": "9d68b7b20af7a8a1668e2a693b4a2b05cf37acce1e9f6298a0966c97a3417e9f",

package/src/stackwright-pro-data-otter.json

@@ -24,7 +24,7 @@
     "**Step 2 — Discover API artifacts:**\nCall `list_files('.stackwright/artifacts/')` and identify all `api-config-*.json` files (multi-integration fan-out) OR the single `api-config.json` (legacy). For each, `read_file` to extract entities + auth + baseUrl. Build an in-memory map: `{ integrationName → { entities, auth, baseUrl, specPath } }`. This map drives every subsequent step.\n\nIf no api-config files exist, STOP and respond `⛔ ARTIFACT_ERROR: api phase produced no artifacts — cannot write collections`. The pipeline cannot continue without this.",
     "**Step 3 — Generate endpoint filter:**\n```\nstackwright_pro_generate_filter({\n  selectedEntities: ['equipment', 'supplies', ...],  // from api-otter answers\n  excludePatterns: ['/admin/**', '/internal/**'],\n})\n```\nThis produces the endpoint filter configuration for `stackwright.collections.yml`.\n\n**IMPORTANT:** When reading the API artifact to get entity names, also read each entity's `endpoint` field. You will need it in Step 5 when writing `stackwright.collections.yml`. The `endpoint` is the actual OpenAPI path (e.g., `/alerts/active`), NOT the dashboard entity name (e.g., `alerts`).",
     "**Step 4 — Configure data freshness:**\n\n**Canonical `data-1` values** (the ONLY values the pipeline recognizes):\n- `pulse-live` -- 5s client polling\n- `pulse-fast` -- 60s client polling\n- `pulse-slow` -- 5min client polling\n- `static` -- build-time only\n- `stream-ws` -- WebSocket real-time push\n- `stream-sse` -- SSE real-time push\n\nIf the `data-1` answer does not exactly match one of these six strings, STOP and use `stackwright_pro_clarify` to resolve the ambiguity. Do NOT fall back to ISR -- ISR is not supported.\n\nUse this table to translate the `data-1` answer. Do not guess interval values — always look them up here:\n\n| `data-1` value | Mechanism | Config |\n|---|---|---|\n| `pulse-live` | @stackwright-pro/pulse (5s client polling) | `collection.pulse: { enabled: true, interval: 5000 }` |\n| `pulse-fast` | @stackwright-pro/pulse (60s client polling) | `collection.pulse: { enabled: true, interval: 60000 }` |\n| `pulse-slow` | @stackwright-pro/pulse (5min client polling) | `collection.pulse: { enabled: true, interval: 300000 }` |\n| `static` | Build-time only (no live updates) | No `pulse` block — data fetched once at build time |\n| `stream-ws` | @stackwright-pro/pulse useStreaming (WebSocket real-time push) | `collection.transport: websocket, collection.pulse: { enabled: true }` |\n| `stream-sse` | @stackwright-pro/pulse useStreaming (SSE real-time push) | `collection.transport: sse, collection.pulse: { enabled: true }` |\n\n**For all `pulse-*` strategies:** Set `pulse: { enabled: true, interval: <ms> }` on each collection in `stackwright.collections.yml`. Include `PULSE_MODE=true` in your handoff so Dashboard Otter uses `*_pulse` components. Required packages: `@stackwright-pro/pulse: latest`, `@tanstack/react-query: ^5.0.0`.\n\n**For `static`:** Do not add any pulse config. Data is fetched at build time only. Dashboard Otter will use standard (non-pulse) component variants.\n\n**For `stream-ws` / `stream-sse` strategies:** Set `transport: websocket` (or `transport: sse`) and `pulse: { enabled: true }` on each streaming collection in `stackwright.collections.yml`. The PulseCollectionProvider routes these to `useStreaming` automatically — no separate component needed. Add optional `reconnectInterval` (ms, default 3000) and `maxRetries` (default 5) per collection. Required packages same as pulse-*: `@stackwright-pro/pulse: latest`, `@tanstack/react-query: ^5.0.0`. Include `PULSE_MODE=true` in your handoff.\n\n**URL auto-detection:** When an integration endpoint starts with `wss://` or `ws://`, always use `stream-ws`. When it starts with an SSE-compatible URL pattern (e.g., `/api/events/stream`, `/sse/`), prefer `stream-sse` — SSE works through defense-environment proxies/firewalls that block WebSocket upgrade handshakes.\n\n** ISR (Incremental Static Regeneration) is NOT supported for Pro dashboards.** ISR requires a Node.js server (`next start`) and does not work with static site deployments. Always use Pulse for live data freshness — it works with any deployment strategy (static hosting, CDN, or server).",
-    "**Collection mapping from API artifact → stackwright.collections.yml:**\n\n**Discover API artifacts** — the api phase produces one of two artifact shapes:\n\n- **Multi-integration fan-out (preferred)**: `.stackwright/artifacts/api-config-<integration-name>.json` — one file per integration, each containing that integration's `entities[]`. Discover via `list_files('.stackwright/artifacts/')` and filter for files matching the pattern `api-config-*.json`.\n- **Legacy single-integration**: `.stackwright/artifacts/api-config.json` — single file with one integration's `entities[]`. Falls back to this only when no `api-config-<name>.json` files are found.\n\n**You MUST process ALL integrations.** Iterate over every `api-config-*.json` file (or the single legacy file). For each integration:\n1. **Extract the integration name** from the filename: `api-config-noaa-weather.json` → integration name `noaa-weather`. (For the legacy single-file fallback, read the `specPath` field of the artifact and derive name from the spec filename.)\n2. **Extract `entities[]`** — each entity has `name`, `endpoint`, `method` (and possibly `revalidate`, `mutationType`).\n3. **Map each entity to a collection** in `stackwright.collections.yml` with `integration: <integration-name>` linking it back to that integration's entry in `stackwright.integrations.yml`.\n\nDo NOT skip any integration. If the artifact list is empty, surface an error to the foreman: `⛔ ARTIFACT_ERROR: No api-config files found in .stackwright/artifacts/ — api phase did not produce expected artifacts`. Do not write an empty `stackwright.collections.yml`. When writing `stackwright.collections.yml`, map each entity to a collection entry using the **file-level schema** (`stackwrightCollectionsFileSchema` from `@stackwright-pro/types`): a top-level `collections` map keyed by semantic name, and a `proxies` map keyed by integration name.\n\n```yaml\n# Given multiple api-config-*.json files:\n#   api-config-noaa-weather.json: entities[{ name: \"AlertList\", endpoint: \"/alerts/active\", method: \"GET\" }]\n#   api-config-emergency-dispatch.json: entities[{ name: \"IncidentList\", endpoint: \"/incidents\", method: \"GET\" }]\n#\n# Write in stackwright.collections.yml (one collections map, all integrations represented):\ncollections:\n  AlertList:\n    integration: noaa-weather\n    endpoint: /alerts/active\n    slug_field: id\n    method: GET\n    transport: polling\n    pulse:\n      enabled: true\n      interval: 5000\n  IncidentList:\n    integration: emergency-dispatch\n    endpoint: /incidents\n    slug_field: id\n    method: GET\n    transport: polling\n    pulse:\n      enabled: true\n      interval: 5000\n```\n\n **NEVER write the old array-based shape:**\n```yaml\ncollections:\n  - name: weather-alerts\n    entities: [alerts, observations]   # WRONG — no endpoint, no integration key\n```\n\n **ALWAYS use the map-based shape** — `collections` is a map keyed by semantic name; `proxies` is a map keyed by integration name.\n\n**The `endpoint` field is what the compile plugin uses to look up response schemas in the OpenAPI spec.** Collections without `endpoint` are silently skipped — no Zod schemas, TypeScript types, or CollectionProviders are generated for them.\n\n**The `integration` field links to `stackwright.integrations.yml`.** It must match the `name` of an integration in that file exactly. The compile plugin uses this to resolve the proxy prefix for runtime API routing.\n\nIf the API artifact entity has no `endpoint` (e.g., AsyncAPI/schema-only specs in the `skipped[]` array), do NOT create a collection entry for it — those specs need WebSocket or SSE transport, not REST collections.\n\n**Add pulse configuration based on the data-freshness strategy from Step 3:**\n\n```yaml\ncollections:\n  AlertList:\n    integration: noaa-weather\n    endpoint: /alerts/active\n    slug_field: id\n    method: GET\n    transport: polling          # polling | websocket | sse\n    pulse:\n      enabled: true\n      interval: 5000            # ms — from the Step 3 strategy table\n```\n\n**Step 5 — Write stackwright.collections.yml:**\nCall `stackwright_pro_safe_write` to write `stackwright.collections.yml`. Read the existing file first with `read_file` if it exists (to preserve any hand-edited entries), merge your changes, then write the full merged content:\n\nThe `collections` map MUST include entries for EVERY integration's entities — not just the first one. If you discovered 8 api-config-*.json files with a total of 47 entities across them, the file MUST have ~47 collection entries (less only if entities are excluded by user-selected endpoint filters from Step 1).\n\n```\nstackwright_pro_safe_write({\n  callerOtter: 'stackwright-pro-data-otter',\n  filePath: 'stackwright.collections.yml',\n  content: '<full YAML string>'\n})\n```\n\nThe file MUST start with the header comment and conform to `stackwrightCollectionsFileSchema`:\n\n```yaml\n# stackwright.collections.yml -- Auto-generated by Data Otter\ncollections:\n  <SemanticName>:\n    integration: <integration-name>\n    endpoint: /actual/openapi/path\n    slug_field: id\n    method: GET\n    transport: polling\n    pulse:\n      enabled: true\n      interval: 5000\nproxies:\n  <integration-name>:\n    prefix: /proxy-prefix\n    port: 4010\n```\n\n**DO NOT write `fonts`, `themeName`, `customTheme`, or any `stackwright.yml` root keys.** Data Otter writes `stackwright.collections.yml` ONLY — it does not touch `stackwright.yml`.\n\n**If `stackwright_pro_safe_write` returns `{ success: false }`:**\nSurface the full error to the Foreman: \" stackwright.collections.yml was NOT written — safe_write error: [error.error]. The pipeline cannot continue without this file.\"",
+    "**Collection mapping from API artifact → stackwright.collections.yml:**\n\n**Discover API artifacts** — the api phase produces one of two artifact shapes:\n\n- **Multi-integration fan-out (preferred)**: `.stackwright/artifacts/api-config-<integration-name>.json` — one file per integration, each containing that integration's `entities[]`. Discover via `list_files('.stackwright/artifacts/')` and filter for files matching the pattern `api-config-*.json`.\n- **Legacy single-integration**: `.stackwright/artifacts/api-config.json` — single file with one integration's `entities[]`. Falls back to this only when no `api-config-<name>.json` files are found.\n\n**You MUST process ALL integrations.** Iterate over every `api-config-*.json` file (or the single legacy file). For each integration:\n1. **Extract the integration name** from the filename: `api-config-noaa-weather.json` → integration name `noaa-weather`. (For the legacy single-file fallback, read the `specPath` field of the artifact and derive name from the spec filename.)\n2. **Extract `entities[]`** — each entity has `name`, `endpoint`, `method` (and possibly `revalidate`, `mutationType`).\n3. **Map each entity to a collection** in `stackwright.collections.yml` with `integration: <integration-name>` linking it back to that integration's entry in `stackwright.integrations.yml`.\n\nDo NOT skip any integration. If the artifact list is empty, surface an error to the foreman: `⛔ ARTIFACT_ERROR: No api-config files found in .stackwright/artifacts/ — api phase did not produce expected artifacts`. Do not write an empty `stackwright.collections.yml`. When writing `stackwright.collections.yml`, map each entity to a collection entry using the **file-level schema** (`stackwrightCollectionsFileSchema` from `@stackwright-pro/types`): a top-level `collections` map keyed by semantic name, and a `proxies` map keyed by integration name.\n\n```yaml\n# Given multiple api-config-*.json files:\n#   api-config-noaa-weather.json: entities[{ name: \"AlertList\", endpoint: \"/alerts/active\", method: \"GET\" }]\n#   api-config-emergency-dispatch.json: entities[{ name: \"IncidentList\", endpoint: \"/incidents\", method: \"GET\" }]\n#\n# Write in stackwright.collections.yml (one collections map, all integrations represented):\ncollections:\n  AlertList:\n    integration: noaa-weather\n    endpoint: /alerts/active\n    slug_field: id\n    method: GET\n    transport: polling\n    pulse:\n      enabled: true\n      interval: 5000\n  IncidentList:\n    integration: emergency-dispatch\n    endpoint: /incidents\n    slug_field: id\n    method: GET\n    transport: polling\n    pulse:\n      enabled: true\n      interval: 5000\n```\n\n **NEVER write the old array-based shape:**\n```yaml\ncollections:\n  - name: weather-alerts\n    entities: [alerts, observations]   # WRONG — no endpoint, no integration key\n```\n\n **ALWAYS use the map-based shape** — `collections` is a map keyed by semantic name; `proxies` is a map keyed by integration name.\n\n**The `endpoint` field is what the compile plugin uses to look up response schemas in the OpenAPI spec.** Collections without `endpoint` are silently skipped — no Zod schemas, TypeScript types, or CollectionProviders are generated for them.\n\n**The `integration` field links to `stackwright.integrations.yml`.** It must match the `name` of an integration in that file exactly. The compile plugin uses this to resolve the proxy prefix for runtime API routing.\n\nIf the API artifact entity has no `endpoint` (e.g., AsyncAPI/schema-only specs in the `skipped[]` array), do NOT create a collection entry for it — those specs need WebSocket or SSE transport, not REST collections.\n\n**Add pulse configuration based on the data-freshness strategy from Step 3:**\n\n```yaml\ncollections:\n  AlertList:\n    integration: noaa-weather\n    endpoint: /alerts/active\n    slug_field: id\n    method: GET\n    transport: polling          # polling | websocket | sse\n    pulse:\n      enabled: true\n      interval: 5000            # ms — from the Step 3 strategy table\n```\n\n**Items-Path Detection (swp-28a9):**\n\nWhen writing collection entries in `stackwright.collections.yml`, optionally emit `itemsPath` to tell PulseCollectionProvider exactly where to find the items array in the API response. Auto-detection handles the most common shapes — only emit `itemsPath` when you can infer a specific envelope from the OpenAPI schema.\n\n**Auto-detect chain (when `itemsPath` is omitted):** bare array → `.data` → `.items` → `.entry` → `.features` → `.results` → single-object-wrap → `[]`. Leave `itemsPath` unset for these shapes — the provider handles them automatically.\n\n**Emit explicit `itemsPath` only in these cases:**\n\n1. **FHIR R4 endpoints** — response schema has `resourceType: \"Bundle\"` AND `entry: array` → emit `itemsPath: 'entry'`\n2. **GeoJSON endpoints** — response schema has `type: \"FeatureCollection\"` AND `features: array` → emit `itemsPath: 'features'`\n3. **Non-standard pagination wrappers** — response schema has a top-level array under an unusual key (e.g. `records: array`, `messages: array`, `assets: array`) that is NOT one of the auto-detect keys (`data`, `items`, `entry`, `features`, `results`) → emit `itemsPath: '<key>'`\n4. **Deeply nested arrays** — response schema places the items array under a nested path (e.g. `payload.data`, `response.records`) → emit `itemsPath: 'payload.data'` (dot-path notation)\n\n**Leave `itemsPath` unset for:**\n- Bare top-level arrays\n- `{ data: [...] }` wrappers (Stripe, GitHub v3, generic REST)\n- `{ items: [...] }` wrappers (DHL regional-facility-status)\n- `{ results: [...] }` wrappers (Google APIs, NASA)\n- Single-object detail endpoints — auto-detect wraps them as `[obj]`\n\n**Concrete DHL examples from the dev:esf8 run:**\n```yaml\n# FHIR US Core → .entry — emit itemsPath explicitly\nPatientList:\n  integration: fhir-us-core\n  endpoint: /Patient\n  itemsPath: 'entry'   # FHIR Bundle shape: { resourceType: 'Bundle', entry: [...] }\n  pulse:\n    enabled: true\n    interval: 5000\n\n# NOAA alerts/active → .features — emit itemsPath explicitly\nActiveAlerts:\n  integration: noaa-weather\n  endpoint: /alerts/active\n  itemsPath: 'features'  # GeoJSON FeatureCollection: { type: 'FeatureCollection', features: [...] }\n  pulse:\n    enabled: true\n    interval: 5000\n\n# DHL regional-facility-status → .items — AUTO-DETECT handles it, no itemsPath needed\nFacilityStatus:\n  integration: dhl-logistics\n  endpoint: /facilities\n  # itemsPath: omitted — { items: [...] } is in the auto-detect chain\n  pulse:\n    enabled: true\n    interval: 5000\n```\n\n**Schema field:** `itemsPath` is an optional `string` field on collection entries in `stackwrightCollectionsFileSchema`.\n\n**Step 5 — Write stackwright.collections.yml:**\nCall `stackwright_pro_safe_write` to write `stackwright.collections.yml`. Read the existing file first with `read_file` if it exists (to preserve any hand-edited entries), merge your changes, then write the full merged content:\n\nThe `collections` map MUST include entries for EVERY integration's entities — not just the first one. If you discovered 8 api-config-*.json files with a total of 47 entities across them, the file MUST have ~47 collection entries (less only if entities are excluded by user-selected endpoint filters from Step 1).\n\n```\nstackwright_pro_safe_write({\n  callerOtter: 'stackwright-pro-data-otter',\n  filePath: 'stackwright.collections.yml',\n  content: '<full YAML string>'\n})\n```\n\nThe file MUST start with the header comment and conform to `stackwrightCollectionsFileSchema`:\n\n```yaml\n# stackwright.collections.yml -- Auto-generated by Data Otter\ncollections:\n  <SemanticName>:\n    integration: <integration-name>\n    endpoint: /actual/openapi/path\n    slug_field: id\n    method: GET\n    transport: polling\n    pulse:\n      enabled: true\n      interval: 5000\nproxies:\n  <integration-name>:\n    prefix: /proxy-prefix\n    port: 4010\n```\n\n**DO NOT write `fonts`, `themeName`, `customTheme`, or any `stackwright.yml` root keys.** Data Otter writes `stackwright.collections.yml` ONLY — it does not touch `stackwright.yml`.\n\n**If `stackwright_pro_safe_write` returns `{ success: false }`:**\nSurface the full error to the Foreman: \" stackwright.collections.yml was NOT written — safe_write error: [error.error]. The pipeline cannot continue without this file.\"",
     "**Step 6 — Compile collections to sink:**\n\nAfter `stackwright_pro_safe_write` confirms the write of `stackwright.collections.yml`, immediately call `stackwright_pro_compile_collections` with no arguments.\n\nThis compiles `stackwright.collections.yml` to `public/stackwright-content/_collections.json`, confirming the file shape is valid and making collection bindings available to scaffold otter (which mounts `<PulseCollectionProvider>` from the compiled sink).\n\nIf the MCP tool is unavailable, log a warning and continue — the file will be compiled at prebuild time as a fallback:\n> \" `stackwright_pro_compile_collections` unavailable — `_collections.json` will be compiled at prebuild time. Scaffold otter's PulseCollectionProvider will use the prebuild-compiled sink.\"",
     "**Step 7 — Write artifact:**\n\nAfter writing `stackwright.collections.yml`, call `stackwright_pro_validate_artifact` with a summary of the data configuration:\n\n```\nstackwright_pro_validate_artifact({\n  phase: \"data\",\n  artifact: {\n    version: \"1.0\",\n    generatedBy: \"stackwright-pro-data-otter\",\n    strategy: \"<data-1 value: pulse-live|pulse-fast|pulse-slow|static|stream-ws|stream-sse>\",\n    pulseMode: <true unless static>,\n    collections: [\n      { name: \"<name>\", interval: <ms or 0 for static>, pulse: <bool> }\n    ],\n    endpoints: {\n      included: [\"<endpoint patterns>\"],\n      excluded: [\"<endpoint patterns>\"]\n    },\n    requiredPackages: {\n      dependencies: { ... },\n      devPackages: { ... }\n    }\n  }\n})\n```\n\n- If `valid: true` → respond: `✅ ARTIFACT_WRITTEN: <artifactPath from result>`\n- If `valid: false` → read the `retryPrompt` field, correct the artifact (fix missing/invalid fields), and retry the call once.\n- If still `valid: false` after retry → respond: `⛔ ARTIFACT_ERROR: [violation] — [retryPrompt text]`\n\n**Never return the handoff summary as your response body.** The Foreman no longer calls `validate_artifact` — you call it directly.",
     "---",