@yottagraph-app/aether-instructions 1.1.27 → 1.1.29

package/commands/build_my_app.md

@@ -102,15 +102,15 @@ Then read these files to understand what's available:
 
 1. `DESIGN.md` -- project vision and current status
 2. `broadchurch.yaml` -- project config (name, gateway URL, etc.)
-3. **The `api` cursor rule** -- this is critical. It describes the Query Server, the platform's primary data source. Build against platform APIs, not external sources.
+3. **The `data` cursor rule** -- this is critical. It describes the Query Server, the platform's primary data source. Build against platform APIs, not external sources.
 4. **`.cursor/skills/`** — Each subdirectory is one skill. List them, open each skill’s entry (usually `SKILL.md`) and follow its structure to learn what is documented (APIs, schemas, helpers, etc.).
 5. `.cursor/rules/` -- scan rule names to know what other patterns are available
 
-**Important: Use the platform's data.** This app runs on the Lovelace platform, which provides a Query Server with entities, news, filings, sentiment, relationships, events, and more. Read the `api` rule and the skills under `.cursor/skills/` to understand what data is available. Use `getSchema()` to discover entity types and properties at runtime.
+**Important: Use the platform's data.** This app runs on the Lovelace platform, which provides a Query Server with entities, news, filings, sentiment, relationships, events, and more. Read the `data` rule and the skills under `.cursor/skills/` to understand what data is available. Use `getSchema()` to discover entity types and properties at runtime.
 
 Key capabilities:
 
-- **Query Server / Elemental API** -- the primary data source. Use `useElementalClient()` from `@yottagraph-app/elemental-api/client`. See the `api` rule.
+- **Query Server / Elemental API** -- the primary data source. Use `useElementalClient()` from `@yottagraph-app/elemental-api/client`. See the `data` rule.
 - **KV storage** -- always available for preferences and lightweight data (see `pref` rule)
 - **Neon Postgres** -- check if `DATABASE_URL` is in `.env` for database access (see `server` rule)
 - **AI agent chat** -- use the `useAgentChat` composable to build a chat UI for deployed agents

package/commands/update_instructions.md

@@ -100,6 +100,29 @@ cp "$TEMP_DIR/package/commands/"* .cursor/commands/ 2>/dev/null || true
 cp -r "$TEMP_DIR/package/skills/"* .cursor/skills/ 2>/dev/null || true
 ```
 
+### Data-mode variant overlay
+
+If this project uses **mcp-only** (or another non-default mode), re-apply the same overlay `init-project.js` uses. Read the saved mode:
+
+```bash
+MODE=$(tr -d '\n' < .cursor/.aether-data-mode 2>/dev/null || echo "api-mcp")
+PKG="$TEMP_DIR/package"
+if [ "$MODE" != "api-mcp" ] && [ -d "$PKG/variants/$MODE/rules" ]; then
+  cp "$PKG/variants/$MODE/rules/"* .cursor/rules/ 2>/dev/null || true
+fi
+if [ "$MODE" != "api-mcp" ] && [ -d "$PKG/variants/$MODE/commands" ]; then
+  cp "$PKG/variants/$MODE/commands/"* .cursor/commands/ 2>/dev/null || true
+fi
+if [ "$MODE" != "api-mcp" ] && [ -d "$PKG/variants/$MODE/skills" ]; then
+  cp -r "$PKG/variants/$MODE/skills/"* .cursor/skills/ 2>/dev/null || true
+fi
+if [ "$MODE" = "mcp-only" ]; then
+  rm -rf .cursor/skills/elemental-api
+fi
+```
+
+If `.cursor/.aether-data-mode` is missing, skip the overlay (defaults to **api-mcp**).
+
 ---
 
 ## Step 6: Write Manifest and Version Marker

package/package.json

@@ -1,11 +1,12 @@
 {
     "name": "@yottagraph-app/aether-instructions",
-    "version": "1.1.27",
+    "version": "1.1.29",
     "description": "Cursor rules, commands, and skills for Aether development",
     "files": [
         "rules",
         "commands",
-        "skills"
+        "skills",
+        "variants"
     ],
     "repository": {
         "type": "git",

package/rules/aether.mdc

@@ -8,7 +8,7 @@ alwaysApply: true
 
 **Structure:** `pages/` (file-based routing), `components/`, `composables/`, `server/api/`, `agents/` (Python ADK), `mcp-servers/` (Python FastMCP).
 
-**Data:** The Query Server (Elemental API) is the primary data source -- entities, news, filings, sentiment, relationships, events. Use `useElementalClient()` from `@yottagraph-app/elemental-api/client`. Do NOT call external APIs for data the platform provides. Discovery-first: use `getSchema()` to discover entity types and properties at runtime. Skill docs: `skills/elemental-api/` (API endpoints) and `skills/data-model/` (Lovelace entity types, properties, relationships, YAML schemas per fetch source; `SKILL.md` first). Run `npm install` if those directories are empty. Lovelace MCP servers may also be configured (see `api` rule) but are optional — check your tool list before assuming they're available.
+**Data:** This app runs on the Lovelace platform -- entities, news, filings, sentiment, relationships, events. See the `data` rule for access patterns and gotchas. Skill docs: `skills/data-model/` (entity types, properties, relationships; `SKILL.md` first). Do NOT call external APIs for data the platform provides.
 
 **Storage:** KV (Upstash Redis) for preferences and lightweight state via `Pref<T>` from `usePrefsStore()`. Neon Postgres for relational data if connected (check `.env` for `DATABASE_URL`).
 
@@ -18,4 +18,4 @@ alwaysApply: true
 
 **First action for a new project:** Run `/build_my_app`.
 
-**Task-specific rules:** `architecture` (project structure, navigation, server routes, agents, MCP), `api` (Elemental API client, schema discovery, gotchas, optional MCP servers), `design` (DESIGN.md workflow, feature docs), `ui` (page templates, layout patterns), `cookbook` (copy-paste UI patterns), `pref` (KV preferences), `branding` (colors, fonts), `server` (Nitro routes, Neon Postgres, server-side Elemental API), `something-broke` (error recovery, build failures).
+**Task-specific rules:** `architecture` (project structure, navigation, server routes, agents, MCP), `data` (Elemental API / data access patterns and gotchas), `cookbook` (copy-paste UI patterns), `cookbook-data` (data-fetching recipes), `design` (DESIGN.md workflow, feature docs), `ui` (page templates, layout patterns), `pref` (KV preferences), `branding` (colors, fonts), `server` (Nitro routes, Neon Postgres), `server-data` (server-side Elemental API from routes), `agents` (ADK agents), `agents-data` (agents calling Elemental API), `something-broke` (error recovery, build failures).

package/rules/agents-data.mdc

@@ -0,0 +1,72 @@
+---
+description: "ADK agents calling the Elemental API via broadchurch_auth and local testing env vars."
+alwaysApply: false
+globs: agents/**
+---
+
+# Agents: Elemental API (Query Server)
+
+## Connecting to the Elemental API
+
+Use `broadchurch_auth` for all Elemental API calls. It lives at
+`agents/broadchurch_auth.py` and is automatically bundled into each agent
+directory at deploy time. It handles:
+
+- Routing through the Broadchurch Portal gateway proxy in production
+  (no direct QS credentials needed — the portal handles Auth0 M2M auth)
+- Authenticating to the proxy with a per-tenant API key from
+  `broadchurch.yaml` (`gateway.qs_api_key`)
+- Falling back to `ELEMENTAL_API_URL` + `ELEMENTAL_API_TOKEN` env vars
+  for local dev
+
+```python
+try:
+    from broadchurch_auth import elemental_client
+except ImportError:
+    from .broadchurch_auth import elemental_client
+
+def get_schema() -> dict:
+    """Get the yottagraph schema."""
+    resp = elemental_client.get("/elemental/metadata/schema")
+    resp.raise_for_status()
+    return resp.json()
+
+def find_entities(expression: str, limit: int = 10) -> dict:
+    """Search for entities."""
+    resp = elemental_client.post("/elemental/find", data={"expression": expression, "limit": str(limit)})
+    resp.raise_for_status()
+    return resp.json()
+```
+
+The try/except is required because the import path differs between local
+dev (`agents/` on sys.path → absolute import) and Agent Engine runtime
+(code packaged inside an ADK module → relative import).
+
+**Do NOT** hardcode URLs or manually handle auth tokens. Do NOT read
+`broadchurch.yaml` directly — `broadchurch_auth` handles all of this.
+
+Key endpoints:
+- `GET /elemental/metadata/schema` — entity types and properties
+- `POST /elemental/find` — search for entities by expression
+- `POST /entities/search` — search for entities by name (batch, scored)
+- `POST /elemental/entities/properties` — get entity property values
+
+Requirements for agents using the Elemental API (add to `requirements.txt`):
+```
+google-auth>=2.20.0
+pyyaml>=6.0
+```
+
+## Local testing: Elemental API env vars
+
+When testing agents locally with `adk web`, for agents that call the Elemental API:
+
+```bash
+export ELEMENTAL_API_URL=https://stable-query.lovelace.ai
+export ELEMENTAL_API_TOKEN=<your-token>
+```
+
+In production, all Elemental API calls go through the Portal Gateway at
+`{gateway_url}/api/qs/{org_id}/...`. The agent sends `X-Api-Key` (from
+`broadchurch.yaml`) and the portal injects its own Auth0 M2M token
+upstream.

package/rules/agents.mdc

@@ -47,56 +47,8 @@ Key rules:
 - Use `google-adk` for the agent framework, `httpx` for HTTP calls
 - Pin dependency versions in `requirements.txt` for reproducible deployments
 
-## Connecting to the Elemental API
-
-Use `broadchurch_auth` for all Elemental API calls. It lives at
-`agents/broadchurch_auth.py` and is automatically bundled into each agent
-directory at deploy time. It handles:
-
-- Routing through the Broadchurch Portal gateway proxy in production
-  (no direct QS credentials needed — the portal handles Auth0 M2M auth)
-- Authenticating to the proxy with a per-tenant API key from
-  `broadchurch.yaml` (`gateway.qs_api_key`)
-- Falling back to `ELEMENTAL_API_URL` + `ELEMENTAL_API_TOKEN` env vars
-  for local dev
-
-```python
-try:
-    from broadchurch_auth import elemental_client
-except ImportError:
-    from .broadchurch_auth import elemental_client
-
-def get_schema() -> dict:
-    """Get the yottagraph schema."""
-    resp = elemental_client.get("/elemental/metadata/schema")
-    resp.raise_for_status()
-    return resp.json()
-
-def find_entities(expression: str, limit: int = 10) -> dict:
-    """Search for entities."""
-    resp = elemental_client.post("/elemental/find", data={"expression": expression, "limit": str(limit)})
-    resp.raise_for_status()
-    return resp.json()
-```
-
-The try/except is required because the import path differs between local
-dev (`agents/` on sys.path → absolute import) and Agent Engine runtime
-(code packaged inside an ADK module → relative import).
-
-**Do NOT** hardcode URLs or manually handle auth tokens. Do NOT read
-`broadchurch.yaml` directly — `broadchurch_auth` handles all of this.
-
-Key endpoints:
-- `GET /elemental/metadata/schema` — entity types and properties
-- `POST /elemental/find` — search for entities by expression
-- `POST /entities/search` — search for entities by name (batch, scored)
-- `POST /elemental/entities/properties` — get entity property values
-
-Requirements for agents using the Elemental API (add to `requirements.txt`):
-```
-google-auth>=2.20.0
-pyyaml>=6.0
-```
+For agents that call the **Elemental API** (`broadchurch_auth`, endpoints,
+local `ELEMENTAL_*` env vars), see the `agents-data` rule.
 
 ## Local Testing
 
@@ -113,10 +65,6 @@ export GOOGLE_CLOUD_PROJECT=broadchurch
 export GOOGLE_CLOUD_LOCATION=us-central1
 export GOOGLE_GENAI_USE_VERTEXAI=1
 
-# For agents that call the Elemental API:
-export ELEMENTAL_API_URL=https://stable-query.lovelace.ai
-export ELEMENTAL_API_TOKEN=<your-token>
-
 adk web                     # Opens browser UI at http://127.0.0.1:8000/dev-ui/
 ```
 
@@ -163,19 +111,20 @@ adk deploy agent_engine \
 
 ## How Agents Reach the App
 
-Once deployed, the agent is reachable through the Portal Gateway:
+Once deployed, the app talks to Agent Engine directly — the portal is only
+in the auth path:
 
 ```
-App (useAgentChat composable)
-  → POST NUXT_PUBLIC_GATEWAY_URL/api/agents/{tenantId}/{agentId}/query
-  → Portal Gateway proxies to Vertex AI Agent Engine (streamQuery)
+Browser → Tenant Nitro Server (POST /api/agent/:agentId/stream)
+  → Portal /authorize (gets short-lived tenant SA token, cached 15 min)
+  → Agent Engine :streamQuery (direct, single hop)
   → Agent runs (may invoke tools, make multiple LLM calls)
-  → Gateway collects the ADK event stream, extracts final text
-  → App displays it
+  → Nitro re-emits ADK events as clean SSE to the browser
 ```
 
 The gateway URL and tenant ID come from `broadchurch.yaml` (injected as
-`NUXT_PUBLIC_GATEWAY_URL` and `NUXT_PUBLIC_TENANT_ORG_ID`).
+`NUXT_PUBLIC_GATEWAY_URL` and `NUXT_PUBLIC_TENANT_ORG_ID`). The token is
+minted for the tenant's GCP service account via SA impersonation.
 
 ### Agent Discovery
 
@@ -207,7 +156,8 @@ Each agent entry has:
 | `engine_id` | `string` | Vertex AI Agent Engine resource ID — used as `{agentId}` in query/stream URLs |
 
 The `engine_id` is the key value — it becomes the `{agentId}` path
-parameter in `POST /api/agents/{tenantId}/{agentId}/query`.
+parameter in `/api/agent/{agentId}/stream` (the local Nitro route) and
+the portal's `/authorize` endpoint.
 
 **How agents get populated:** The portal discovers agents from two sources:
 1. **Firestore** — agents registered by the deploy workflow (`deploy-agent.yml`
@@ -232,39 +182,24 @@ const agents = config.value?.agents ?? [];
 const agentId = agents[0]?.engine_id; // use as {agentId} in query URLs
 ```
 
-### Gateway Request
-
-```
-POST {NUXT_PUBLIC_GATEWAY_URL}/api/agents/{tenantId}/{agentId}/query
-Content-Type: application/json
-
-{ "message": "Summarize the latest 8-K filing", "session_id": "optional-session-id" }
-```
+### Streaming via the Local Nitro Route
 
-Omit `session_id` on the first message — the gateway auto-creates one.
+The `useAgentChat` composable calls `POST /api/agent/:agentId/stream` (the
+local Nitro route). This route handles token acquisition, session creation,
+and SSE parsing — the browser just sees clean Server-Sent Events.
 
-### Gateway Response Format
+For custom server-side agent calls (e.g. background tasks), you can call the
+portal's `/authorize` endpoint directly and then use the token against
+Agent Engine:
 
-```json
-{
-  "output": "The agent's final text response",
-  "session_id": "session-abc-123",
-  "events": [ /* raw ADK event stream */ ]
-}
+```typescript
+const auth = await $fetch(`${gatewayUrl}/api/agents/${orgId}/${agentId}/authorize`, {
+    method: 'POST',
+    body: { user_id: 'background-task', create_session: false },
+});
+// auth.token, auth.engine_url, auth.expires_in
 ```
 
-| Field | Type | Description |
-|---|---|---|
-| `output` | `string \| any[]` | Usually the agent's final text. Falls back to the raw `events` array if the gateway couldn't extract text. |
-| `session_id` | `string` | Pass back on subsequent messages to continue the conversation. |
-| `events` | `any[]` | Full ADK event stream. Useful for debugging or building UIs that show intermediate agent steps. |
-
-**Important:** `output` is NOT always a string. When the agent's response
-involves complex tool chains, the gateway's server-side extraction may miss
-the text and return the raw events array instead. Always use
-`extractAgentText()` to parse `output` safely rather than treating it as a
-string directly.
-
 ### ADK Event Stream Format
 
 The `events` array contains one object per step the agent took. Each event
@@ -292,13 +227,11 @@ than objects — always handle both.
 
 ### Parsing Agent Responses (Non-Streaming)
 
-For one-shot calls (server routes, background tasks) where streaming isn't
-needed, use the buffered `/query` endpoint with `extractAgentText`:
+For non-streaming use cases (e.g. extracting text from a buffered response),
+use `extractAgentText`:
 
 ```typescript
 import { extractAgentText } from '~/composables/useAgentChat';
-
-const response = await $fetch(url, { method: 'POST', body: { message } });
 const text = extractAgentText(response.output);
 ```
 
@@ -307,28 +240,20 @@ JSON-string or object elements), single event objects, and several legacy
 Agent Engine response shapes. It skips `functionCall` / `functionResponse`
 events and returns the agent's final text.
 
-### Streaming Responses
+### Streaming SSE Events
 
-The gateway also exposes a streaming endpoint that returns Server-Sent
-Events as the agent executes. **The `useAgentChat` composable uses this by
-default** — it tries `/stream` first and falls back to `/query`
+The local Nitro route (`POST /api/agent/:agentId/stream`) returns
+Server-Sent Events. The `useAgentChat` composable handles these
 automatically.
 
-```
-POST {NUXT_PUBLIC_GATEWAY_URL}/api/agents/{tenantId}/{agentId}/stream
-Content-Type: application/json
-
-{ "message": "Summarize the latest 8-K filing", "session_id": "optional" }
-```
-
-The response is an SSE stream with these event types:
+SSE event types:
 
 | Event | Data Shape | Description |
 |---|---|---|
 | `text` | `{ "text": "..." }` | Agent text output (replaces previous text) |
 | `function_call` | `{ "name": "...", "args": {...} }` | Agent is calling a tool |
 | `function_response` | `{ "name": "...", "response": {...} }` | Tool returned a result |
-| `error` | `{ "message": "..." }` | Error during processing |
+| `error` | `{ "message": "...", "code": "..." }` | Error during processing (`code` is `PERMISSION_DENIED` for IAM failures) |
 | `done` | `{ "session_id": "...", "text": "..." }` | Stream complete with final text |
 
 For custom agent UIs that need streaming, import `readSSE`:

package/rules/architecture.mdc

@@ -9,7 +9,7 @@ Aether is an app framework built on Nuxt 3 + Vue 3 + Vuetify 3 + TypeScript. It
 
 **Tech stack**: Nuxt 3 (SPA), Vue 3 Composition API (`<script setup>`), Vuetify 3, TypeScript (required), Auth0 (automatic).
 
-**Data source**: This app runs on the Lovelace platform. The Query Server (Elemental API) is the primary data source -- use it for entities, news, filings, sentiment, relationships, and events. See the `api` rule. Do not call external APIs for data the platform provides.
+**Data source**: This app runs on the Lovelace platform (entities, news, filings, sentiment, relationships, events). See the `data` rule for how your app accesses that data. Do not call external APIs for data the platform provides.
 
 ## Project Structure
 
@@ -28,11 +28,10 @@ mcp-servers/        # MCP servers (Python, deploy to Cloud Run)
 
 | Store | Purpose | When to use |
 |---|---|---|
-| Query Server (Elemental API) | Lovelace Knowledge Graph | Entities, news, relationships, events, sentiment, filings |
+| Platform data (Query Server / Postgres per your architecture) | Lovelace knowledge graph or synced local data | See the `data` rule |
 | KV (Upstash Redis) | User preferences, lightweight state | Settings, watchlists, UI state that should persist |
 | Neon Postgres | App-specific relational data | Custom tables, complex queries (if connected — check `DATABASE_URL` in `.env`) |
 
-Use `useElementalClient()` for Query Server data (see `api` rule).
 Use `Pref<T>` or `usePrefsStore()` for KV (see `pref` rule).
 Check `.env` for `DATABASE_URL` before using Neon Postgres (see `server` rule).
 
@@ -113,12 +112,12 @@ using them later when adding features to an established app.
 ## Server Routes
 
 Nitro server routes live in `server/api/` and deploy with the app to Vercel.
-Use server routes when you need to proxy external APIs (avoid CORS), call
-the Elemental API server-side, or keep secrets off the client. Call them
-from client code with `$fetch('/api/my-data/fetch')`.
+Use server routes when you need to proxy external APIs (avoid CORS), access
+data server-side, or keep secrets off the client. Call them from client code
+with `$fetch('/api/my-data/fetch')`.
 
-See the `server` rule for file-routing conventions, Neon Postgres patterns,
-and server-side Elemental API access via `useRuntimeConfig()`.
+See the `server` rule for file-routing conventions and Neon Postgres patterns.
+See `server-data` when calling the platform Query Server from server routes.
 
 ## Beyond the UI: Agents, MCP Servers, and Server Routes
 
@@ -126,7 +125,21 @@ Aether apps are more than just a Nuxt SPA. The project contains three additional
 
 ### `agents/` -- ADK Agents (Python)
 
-Each subdirectory is a self-contained Python agent that deploys to Vertex AI Agent Engine. See the `agents` cursor rule for development patterns. Agents are deployed via the Broadchurch Portal UI or the `/deploy_agent` Cursor command, both of which trigger `deploy-agent.yml`. Once deployed, agents are reachable through the Portal Gateway (`NUXT_PUBLIC_GATEWAY_URL`). Use the `useAgentChat` composable to build a chat UI that talks to them.
+Each subdirectory is a self-contained Python agent that deploys to Vertex AI Agent Engine. See the `agents` cursor rule for development patterns. Agents are deployed via the Broadchurch Portal UI or the `/deploy_agent` Cursor command, both of which trigger `deploy-agent.yml`. Use the `useAgentChat` composable to build a chat UI that talks to them.
+
+**Agent query path:** The app talks to Agent Engine directly — the portal is
+only in the auth path. The flow is:
+
+1. Tenant Nitro route (`server/api/agent/[agentId]/stream.post.ts`) calls the
+   Portal Gateway's `/authorize` endpoint to get a short-lived (15 min) GCP
+   access token minted for the tenant's service account.
+2. The Nitro route uses that token to call Agent Engine's `:streamQuery`
+   directly — streaming data does NOT flow through the portal.
+3. Tokens are cached server-side for their TTL.
+
+If you see a 403 from Agent Engine, the tenant's service account is missing
+IAM permissions (`roles/aiplatform.user`). Check the Broadchurch portal's
+project detail page for IAM health status.
 
 ### `mcp-servers/` -- MCP Servers (Python)
 
@@ -141,14 +154,15 @@ Tenant-specific configuration generated during provisioning. Contains GCP projec
 The template ships composables for interacting with the Lovelace platform.
 Use these to build whatever UI fits the app:
 
-- **`useAgentChat()`** -- Send messages to deployed ADK agents through the
-  Portal Gateway. Handles streaming, session management, and response
-  parsing. See the `agents` cursor rule for gateway endpoints and response
-  formats.
+- **`useAgentChat()`** -- Send messages to deployed ADK agents. Uses the
+  local Nitro streaming route (`/api/agent/:agentId/stream`) which calls
+  Agent Engine directly with a tenant SA token. Handles streaming, session
+  management, and response parsing. Shows clear error messages when IAM
+  permissions are missing. See the `agents` cursor rule for details.
 - **`useTenantConfig()`** -- Fetch the tenant's runtime config (deployed
   agents, feature flags, MCP servers) from the Portal Gateway.
-- **`useElementalClient()`** -- Query Server client for entities, news,
-  filings, etc. See the `api` rule.
+- **`useElementalClient()`** -- When using the Elemental API from the client,
+  see the `data` rule.
 - **`usePrefsStore()` / `Pref<T>`** -- KV-backed user preferences. See
   the `pref` rule.