@yottagraph-app/aether-instructions 1.1.21 → 1.1.23

package/commands/build_my_app.md

@@ -92,7 +92,47 @@ Key capabilities:
 
 ---
 
-## Step 4: Design the UX
+## Step 4: Verify Data Availability
+
+Before designing UX, verify that the data your app needs actually exists
+in the knowledge graph. This prevents building features around empty data.
+
+**If MCP tools are available:**
+
+```
+elemental_get_schema()                          → list entity types, confirm your target types exist
+elemental_get_entity(entity="Microsoft")        → verify entity lookup works with a known entity
+elemental_get_entity(entity="Apple Inc")        → try another known entity
+```
+
+If schema calls succeed but entity lookups return "not found," that means
+the entity type exists in the schema but has no data. That's a **data
+issue**, not a broken server. Try different, well-known entity names.
+
+**If MCP tools are NOT available, use curl:**
+
+```bash
+# Read credentials from broadchurch.yaml
+GW=$(grep 'url:' broadchurch.yaml | head -1 | sed 's/.*"\(.*\)".*/\1/')
+ORG=$(grep 'org_id:' broadchurch.yaml | sed 's/.*"\(.*\)".*/\1/')
+KEY=$(grep 'qs_api_key:' broadchurch.yaml | sed 's/.*"\(.*\)".*/\1/')
+
+# List entity types
+curl -s "$GW/api/qs/$ORG/elemental/metadata/schema" -H "X-Api-Key: $KEY"
+
+# Search for a known entity
+curl -s "$GW/api/qs/$ORG/entities/search" \
+  -X POST -H "Content-Type: application/json" -H "X-Api-Key: $KEY" \
+  -d '{"queries":[{"queryId":1,"query":"Microsoft"}],"maxResults":3,"includeNames":true}'
+```
+
+**Document what you find.** If certain entity types have sparse data, note
+it in your UX plan. Design features around data that actually exists, and
+mark aspirational features (that need more data) as future work.
+
+---
+
+## Step 5: Design the UX
 
 Based on the brief, think about the right UX for this specific problem. Do NOT default to a sidebar-with-tabs layout. Consider:
 
@@ -116,7 +156,7 @@ Present the plan to the user and ask for approval before proceeding.
 
 ---
 
-## Step 5: Build
+## Step 6: Build
 
 Implement the plan:
 
@@ -128,6 +168,13 @@ Implement the plan:
 6. Use Vuetify components and the project's dark theme
 7. Update `DESIGN.md` with what you built
 
+**Use the pre-built platform utilities:**
+
+- `useElementalSchema()` — schema discovery with caching, flavor/PID lookup helpers
+- `buildGatewayUrl()`, `getApiKey()`, `padNeid()` from `utils/elementalHelpers`
+- `searchEntities()`, `getEntityName()` from `utils/elementalHelpers`
+- `useElementalClient()` from `@yottagraph-app/elemental-api/client`
+
 **Follow the project's coding conventions:**
 
 - `<script setup lang="ts">` for all Vue components
@@ -136,7 +183,7 @@ Implement the plan:
 
 ---
 
-## Step 6: Verify
+## Step 7: Verify
 
 After building, check dependencies are installed and run a build:
 
@@ -151,7 +198,7 @@ Then suggest the user run `npm run dev` to preview their app locally.
 
 ---
 
-## Step 7: Next Steps
+## Step 8: Next Steps
 
 > Your app is taking shape! Here's what you can do next:
 >

package/package.json

@@ -1,6 +1,6 @@
 {
     "name": "@yottagraph-app/aether-instructions",
-    "version": "1.1.21",
+    "version": "1.1.23",
     "description": "Cursor rules, commands, and skills for Aether development",
     "files": [
         "rules",

package/rules/api.mdc

@@ -28,13 +28,55 @@ For Lovelace **entity types, properties, relationships, and per-source schemas**
 
 ## Test Before You Build
 
-**ALWAYS test API calls via curl before writing code that depends on them.**
-This is not optional — the Elemental API has response shapes that differ from
-what the TypeScript types suggest, and assumptions about nesting, property
-formats, and field names will be wrong without testing. This applies doubly
-to server-side code, where you can't inspect responses in the browser console.
+**ALWAYS test data access before writing application code.** The Elemental
+API has response shapes that differ from what the TypeScript types suggest,
+and assumptions about nesting, property formats, and field names will be
+wrong without testing.
 
-### How to test
+### Step 1: MCP tools (interactive exploration)
+
+**If MCP tools appear in your tool list, start here.** MCP handles entity
+resolution, PID lookups, and NEID formatting automatically — use it to
+verify what data exists and how it's structured.
+
+```
+elemental_get_schema()                          → list all entity types
+elemental_get_schema(flavor="article")          → properties for a type
+elemental_get_entity(entity="Apple")            → resolve + fetch entity
+elemental_get_related(entity="Apple",
+    related_flavor="person")                    → follow relationships
+```
+
+MCP tells you the correct flavor IDs, property IDs, and data shapes. Use
+these to inform your REST implementation.
+
+**Verify MCP is working with known-good queries:**
+
+```
+elemental_get_schema()                          → should return flavors + properties
+elemental_get_entity(entity="Microsoft")        → should resolve to a company
+elemental_get_entity(entity="Apple Inc")        → another known entity
+elemental_health()                              → server health check
+```
+
+**Interpreting MCP errors — do NOT assume the server is broken:**
+
+- `entity not found` or 404 in entity lookup → the entity doesn't exist
+  in the knowledge graph, not a connectivity problem. Try a different entity.
+- `failed to get property values: 404` → the entity was resolved but has
+  no data for those properties. The MCP server is working correctly.
+- Schema calls succeed but entity calls fail → data is sparse for that
+  entity type. Try well-known entities (Microsoft, Apple Inc, JPMorgan).
+- If `elemental_health()` fails → actual connectivity problem.
+
+**Key insight:** A 404 from an MCP entity/property call means "not found,"
+not "server broken." Always test with known entities before concluding
+the server is down.
+
+### Step 2: curl (verify exact request/response shapes)
+
+MCP doesn't cover every REST endpoint (e.g. `/elemental/find` expressions).
+Test those with curl before implementing them in code.
 
 The gateway proxy authenticates on your behalf — no Auth0 tokens needed.
 Read `broadchurch.yaml` for the three values you need:
@@ -48,8 +90,6 @@ Read `broadchurch.yaml` for the three values you need:
 Build the request URL as `{gateway.url}/api/qs/{tenant.org_id}/{endpoint}`
 and include the header `X-Api-Key: {gateway.qs_api_key}`.
 
-### Example calls
-
 ```bash
 # Variables — read these from broadchurch.yaml
 GW="https://broadchurch-portal-194773164895.us-central1.run.app"
@@ -62,6 +102,13 @@ curl -s "$GW/api/qs/$ORG/entities/search" \
   -H "X-Api-Key: $KEY" \
   -d '{"queries":[{"queryId":1,"query":"Microsoft"}],"maxResults":3}'
 
+# Test a find expression
+curl -s -X POST "$GW/api/qs/$ORG/elemental/find" \
+  -H "X-Api-Key: $KEY" \
+  -H "Content-Type: application/x-www-form-urlencoded" \
+  --data-urlencode 'expression={"type":"is_type","is_type":{"fid":12}}' \
+  --data-urlencode 'limit=5'
+
 # Get entity properties (form-encoded)
 curl -s -X POST "$GW/api/qs/$ORG/elemental/entities/properties" \
   -H "X-Api-Key: $KEY" \
@@ -74,6 +121,47 @@ curl -s -X POST "$GW/api/qs/$ORG/elemental/entities/properties" \
 `application/x-www-form-urlencoded` with JSON-stringified parameter values.
 All other endpoints accept `application/json`.
 
+**Interpreting errors:** 400 = expression syntax is wrong. 500 = expression
+is valid but the query failed (wrong PID, unsupported operator for that
+property type). 200 + empty `eids` = query worked but no results match.
+404 from entity/property endpoints = entity or data doesn't exist (not a
+server error). Always test with known entities (e.g. search for "Microsoft")
+before assuming the API is broken.
+
+### Step 3: Implement with confidence
+
+Now write your composable or server route, knowing the exact API shapes.
+
+## Pre-Built Helpers
+
+The template includes composables and utilities that handle common
+Elemental API patterns. **Use these instead of writing from scratch:**
+
+### `useElementalSchema()` — Schema Discovery with Caching
+
+```typescript
+const { flavors, properties, flavorByName, pidByName, refresh } = useElementalSchema();
+await refresh();                                // fetches once, then cached
+const articleFid = flavorByName('article');      // → number | null
+const namePid = pidByName('name');              // → typically 8
+```
+
+Handles the dual response shapes (`res.schema.flavors` vs `res.flavors`)
+and the `fid`/`findex` naming inconsistency automatically.
+
+### `utils/elementalHelpers` — Gateway URL Helpers
+
+```typescript
+import { buildGatewayUrl, getApiKey, padNeid, searchEntities, getEntityName } from '~/utils/elementalHelpers';
+
+const url = buildGatewayUrl('entities/search');   // full gateway URL
+const key = getApiKey();                          // from runtimeConfig
+const neid = padNeid('4926132345040704022');       // → "04926132345040704022"
+
+const results = await searchEntities('Microsoft'); // batch name search
+const name = await getEntityName(neid);            // display name lookup
+```
+
 ## Client Usage
 
 All API calls go through `useElementalClient()` from `@yottagraph-app/elemental-api/client`.
@@ -163,26 +251,46 @@ knowledge of what's in the graph.
 
 ## API Gotchas
 
-### `getSchema()` response is nested — WILL crash if you don't handle it
+### `getSchema()` response structure differs by endpoint
+
+There are two schema endpoints with **different response shapes**:
 
-The generated TypeScript types suggest `response.properties` and
-`response.flavors` exist at the top level. **They don't.** The API nests
-them under `response.schema`. This mismatch between types and reality
-causes `Cannot read properties of undefined` every time.
+| Endpoint | Flavors at | Flavor ID field | Detail level |
+|----------|-----------|-----------------|--------------|
+| `GET /schema` | top-level (`res.flavors`) | `findex` | Rich (display names, units, domains) |
+| `GET /elemental/metadata/schema` | nested (`res.schema.flavors`) | `fid` | Basic (name + type only) |
+
+The TypeScript client's `getSchema()` calls `/elemental/metadata/schema`,
+so the response nests data under `.schema`. The generated types may suggest
+top-level access, but it won't work at runtime.
 
 ```typescript
-// WRONG — will crash at runtime despite TypeScript compiling fine:
+// WRONG — will crash (data is nested under .schema):
 const res = await client.getSchema();
 const props = res.properties; // undefined!
 
-// CORRECT — always access through .schema:
+// CORRECT — always use fallback to handle both shapes:
 const res = await client.getSchema();
 const properties = res.schema?.properties ?? (res as any).properties ?? [];
 const flavors = res.schema?.flavors ?? (res as any).flavors ?? [];
 ```
 
-The `(res as any).properties` fallback is there in case the API is ever
-fixed to match the types. Use this pattern every time.
+### Flavor ID field: `fid` vs `findex`
+
+The flavor identifier has **different field names** depending on the endpoint:
+`GET /schema` returns `findex`, `/elemental/metadata/schema` returns `fid`.
+Same value, different key. Always use a fallback:
+
+```typescript
+const articleFlavor = flavors.find(f => f.name === 'article');
+const articleFid = articleFlavor?.fid ?? articleFlavor?.findex ?? null;
+
+// When building a FID lookup map:
+const fidMap = new Map(flavors.map(f => [f.fid ?? f.findex, f.name]));
+```
+
+The `is_type` expression in `/elemental/find` always uses the `fid` key
+regardless of which schema endpoint provided the value.
 
 ### Relationship property values need zero-padding to form valid NEIDs
 
@@ -273,6 +381,23 @@ const docNeids = (res.values ?? []).map((v) => String(v.value).padStart(20, '0')
 
 See the **cookbook** rule for a full "Get filings for a company" recipe.
 
+### Expression language pitfalls
+
+These mistakes come up repeatedly when building `/elemental/find` queries:
+
+- **Entity type filtering**: Use `is_type` (not `comparison` with pid=0).
+  `comparison` requires `pid != 0`.
+- **`string_like` is name-only**: Only works on the name property (PID 8).
+  Use `eq` for exact matches on other string properties.
+- **Boolean combinators**: Use `{"type": "and", "and": [...]}` — not
+  `conjunction` or any other name.
+- **`lt`/`gt` are numeric-only**: Only work on `data_int` and `data_float`
+  properties.
+- **`regex` is not implemented**: Will return an error.
+
+Read the "Common Mistakes" section in the **elemental-api skill** (`find.md`)
+for examples of each.
+
 ### Entity Search
 
 Use `client.findEntities()` (`POST /elemental/find`) for entity search.
@@ -327,17 +452,13 @@ const response = await getArticle(artid);
 if (response.status === 404) { /* handle not found */ }
 ```
 
-## Lovelace MCP Servers (Optional)
+## Lovelace MCP Servers
 
-Four MCP servers **may** be configured in `.cursor/mcp.json` for interactive
-data exploration. They are NOT required — the REST client
-(`useElementalClient()`) and skill docs cover the same data and are the
-primary path for building features.
-
-**Before referencing MCP tools, check your available tool list.** If tools
-like `elemental_get_schema` don't appear in your MCP server list, the
-servers aren't connected — just use the REST client and skill docs instead.
-Do not report this as a problem; it's a normal configuration state.
+Four MCP servers may be configured in `.cursor/mcp.json` for interactive
+data exploration. **Check your tool list** — if tools like
+`elemental_get_schema` appear, use them as your primary testing and
+exploration interface before writing code. If they don't appear, the
+servers aren't connected; use curl and the skill docs instead.
 
 | Server | What it provides |
 |---|---|
@@ -346,6 +467,23 @@ Do not report this as a problem; it's a normal configuration state.
 | `lovelace-wiki` | Wikipedia entity enrichment |
 | `lovelace-polymarket` | Prediction market data |
 
+### MCP Tool Quick Reference
+
+| Tool | Purpose | Use to verify... |
+|---|---|---|
+| `elemental_get_schema` | Discover entity types (flavors), properties, and relationships | Flavor IDs, property IDs, data types |
+| `elemental_get_entity` | Look up entity by name or NEID; returns properties | Entity resolution, property shapes |
+| `elemental_get_related` | Related entities with type/relationship filters | Relationship types and traversal |
+| `elemental_get_relationships` | Relationship types and counts between two entities | Edge types between specific entities |
+| `elemental_graph_neighborhood` | Most influential neighbors of an entity | Graph connectivity |
+| `elemental_graph_sentiment` | Sentiment analysis from news articles | Sentiment data availability |
+| `elemental_get_events` | Events for an entity or by search query | Event categories and shapes |
+| `elemental_health` | Health check | Server connectivity |
+
+MCP tools handle entity resolution, PID lookups, and NEID formatting
+automatically. Use them to discover IDs and verify data exists before
+building REST-based features with `useElementalClient()`.
+
 ### Setup
 
 `.cursor/mcp.json` is auto-generated by `init-project.js`. If it's missing,
@@ -353,25 +491,3 @@ run `node init-project.js --local` to regenerate it. For provisioned projects,
 the servers route through the Portal Gateway proxy (no credentials needed).
 For local development without a gateway, the servers require an
 `AUTH0_M2M_DEV_TOKEN` environment variable.
-
-These servers are also accessible from the browser through the Portal
-Gateway, so you can build MCP tool exploration UIs if needed.
-
-### When MCP Servers Are Available
-
-If the MCP tools appear in your tool list, you can use them for interactive
-exploration alongside the REST client:
-
-| Tool | Purpose |
-|---|---|
-| `elemental_get_schema` | Discover entity types (flavors), properties, and relationships |
-| `elemental_get_entity` | Look up entity by name or NEID; returns properties |
-| `elemental_get_related` | Related entities with type/relationship filters |
-| `elemental_get_relationships` | Relationship types and counts between two entities |
-| `elemental_graph_neighborhood` | Most influential neighbors of an entity |
-| `elemental_graph_sentiment` | Sentiment analysis from news articles |
-| `elemental_get_events` | Events for an entity or by search query |
-| `elemental_health` | Health check |
-
-MCP is convenient for schema discovery and entity resolution during planning.
-For building UI features, always use the REST client (`useElementalClient()`).

package/rules/cookbook.mdc

@@ -388,7 +388,169 @@ Two-column layout with selectable list and detail panel.
 </script>
 ```
 
-## 7. Get Filings for a Company
+## 7. News Feed — Recent Articles with Sentiment
+
+Fetch recent articles from the knowledge graph. Uses `useElementalSchema()`
+for runtime flavor/PID discovery and `buildGatewayUrl()` for gateway access.
+
+```vue
+<template>
+    <div class="d-flex flex-column fill-height pa-4">
+        <h1 class="text-h5 mb-4">Recent News</h1>
+        <v-alert v-if="error" type="error" variant="tonal" class="mb-4" closable>
+            {{ error }}
+        </v-alert>
+        <v-progress-linear v-if="loading" indeterminate class="mb-4" />
+        <v-list v-if="articles.length" lines="three">
+            <v-list-item v-for="a in articles" :key="a.neid">
+                <template #title>
+                    <span>{{ a.name || a.neid }}</span>
+                    <v-chip
+                        v-if="a.sentiment"
+                        size="x-small"
+                        class="ml-2"
+                        :color="a.sentiment > 0 ? 'success' : a.sentiment < 0 ? 'error' : 'grey'"
+                    >
+                        {{ a.sentiment > 0 ? 'Bullish' : a.sentiment < 0 ? 'Bearish' : 'Neutral' }}
+                    </v-chip>
+                </template>
+                <template #subtitle>{{ a.neid }}</template>
+            </v-list-item>
+        </v-list>
+        <v-empty-state
+            v-else-if="!loading"
+            headline="No articles found"
+            icon="mdi-newspaper-variant-outline"
+        />
+    </div>
+</template>
+
+<script setup lang="ts">
+    import { useElementalClient } from '@yottagraph-app/elemental-api/client';
+    import { padNeid } from '~/utils/elementalHelpers';
+
+    const client = useElementalClient();
+    const { flavorByName, pidByName, refresh: loadSchema } = useElementalSchema();
+
+    const articles = ref<{ neid: string; name: string; sentiment: number | null }[]>([]);
+    const loading = ref(false);
+    const error = ref<string | null>(null);
+
+    onMounted(async () => {
+        loading.value = true;
+        try {
+            await loadSchema();
+            const articleFid = flavorByName('article');
+            if (!articleFid) {
+                error.value = 'Article entity type not found in schema';
+                return;
+            }
+
+            const res = await client.findEntities({
+                expression: JSON.stringify({ type: 'is_type', is_type: { fid: articleFid } }),
+                limit: 20,
+            });
+            const neids: string[] = (res as any).eids ?? [];
+
+            if (!neids.length) {
+                return;
+            }
+
+            const namePid = pidByName('name');
+            const sentimentPid = pidByName('sentiment');
+            const pids = [namePid, sentimentPid].filter((p): p is number => p !== null);
+
+            const props = await client.getPropertyValues({
+                eids: JSON.stringify(neids),
+                pids: JSON.stringify(pids),
+            });
+
+            const valueMap = new Map<string, Record<number, any>>();
+            for (const v of (props as any).values ?? []) {
+                const eid = padNeid(v.eid ?? v.entity_id ?? '');
+                if (!valueMap.has(eid)) valueMap.set(eid, {});
+                valueMap.get(eid)![v.pid] = v.value;
+            }
+
+            articles.value = neids.map((neid) => {
+                const vals = valueMap.get(neid) ?? {};
+                return {
+                    neid,
+                    name: namePid ? (vals[namePid] as string) ?? neid : neid,
+                    sentiment: sentimentPid ? (vals[sentimentPid] as number) ?? null : null,
+                };
+            });
+        } catch (e: any) {
+            error.value = e.message || 'Failed to load articles';
+        } finally {
+            loading.value = false;
+        }
+    });
+</script>
+```
+
+## 8. Entity Search with Gateway Helpers
+
+Simpler version of recipe #1 using the pre-built `searchEntities()` helper.
+
+```vue
+<template>
+    <div class="d-flex flex-column fill-height pa-4">
+        <h1 class="text-h5 mb-4">Entity Search</h1>
+        <v-text-field
+            v-model="query"
+            label="Search entities"
+            prepend-inner-icon="mdi-magnify"
+            variant="outlined"
+            @keyup.enter="search"
+            :loading="loading"
+        />
+        <v-alert v-if="error" type="error" variant="tonal" class="mt-2" closable>
+            {{ error }}
+        </v-alert>
+        <v-list v-if="results.length" class="mt-4">
+            <v-list-item
+                v-for="r in results"
+                :key="r.neid"
+                :title="r.name"
+                :subtitle="r.neid"
+            />
+        </v-list>
+        <v-empty-state
+            v-else-if="searched && !loading"
+            headline="No results"
+            icon="mdi-magnify-remove-outline"
+        />
+    </div>
+</template>
+
+<script setup lang="ts">
+    import { searchEntities } from '~/utils/elementalHelpers';
+
+    const query = ref('');
+    const results = ref<{ neid: string; name: string }[]>([]);
+    const loading = ref(false);
+    const error = ref<string | null>(null);
+    const searched = ref(false);
+
+    async function search() {
+        if (!query.value.trim()) return;
+        loading.value = true;
+        error.value = null;
+        searched.value = true;
+        try {
+            results.value = await searchEntities(query.value.trim());
+        } catch (e: any) {
+            error.value = e.message || 'Search failed';
+            results.value = [];
+        } finally {
+            loading.value = false;
+        }
+    }
+</script>
+```
+
+## 9. Get Filings for a Company
 
 Fetch Edgar filings (or any relationship-linked documents) for an organization.
 Uses `$fetch` for the initial entity search because `POST /entities/search`