@lunora/cli 1.0.0-alpha.37 → 1.0.0-alpha.371

package/skills/lunora-performance-audit/SKILL.md

@@ -1,143 +1,185 @@
 ---
 name: lunora-performance-audit
-description: Diagnoses and fixes Lunora performance problems — full-table scans, missing
-    indexes, OCC write conflicts, oversized subscriptions, and sharding/`.global()`
-    scaling. Use when queries are slow, mutations conflict, or `@lunora/advisor`
-    flags a table.
+description: Diagnoses and fixes Lunora performance problems — full-table scans and missing indexes, expensive `ctx.db.related` traversals, OCC write conflicts (409 `CONFLICT`), live queries that re-run too often, hot shards, and choosing `.shardBy(key)` or `.global()` to scale. Starts from measured signal (`lunora insights`) and static signal (`lunora advisor`, `@lunora/advisor` lints such as `filter_without_index`, `unbounded_collect`, `unindexed_foreign_key`, `hot_shard`). Use when the user says a query or page is slow, mutations fail with conflicts under load, subscriptions are chatty, asks to "audit performance", "add the right index", or "should I shard this table", or when the Studio Advisors tab or `lunora advisor` flags something.
 ---
 
 # Lunora Performance Audit
 
-Systematically diagnose Lunora performance issues, route to the right fix, and
-apply it across sibling functions consistently.
+Find the measured problem, route it to the matching fix below, and apply the
+fix to every function that touches the same table.
 
-## When to Use
+If traffic is modest and nothing is measured as slow, prefer simpler code. Don't
+introduce sharding, digest tables or document splits on speculation.
 
-- Queries feel slow or read far more rows than they return.
-- Mutations retry or conflict under load (OCC).
-- Subscriptions fan out updates too broadly or re-run too often.
-- The Lunora Studio **Advisors** tab or `@lunora/advisor` flags a table.
+## Workflow
 
-## When Not to Use
+1. **Scope** one concrete user flow with a clear entry and exit.
+2. **Gather signal** with `lunora insights` (runtime) and `lunora advisor`
+   (static).
+3. **Trace** every `ctx.db` read and write in that flow.
+4. **Route** to the problem class below and fix it.
+5. **Fix siblings** so every function reads the table the same indexed way. A
+   half-migrated access pattern is its own bug.
+6. **Verify:** behaviour unchanged, the advisor finding cleared, and the
+   `insights` row improved after the fix has served traffic.
+
+## Signal
+
+### Runtime: `lunora insights`
+
+Needs a running worker that has served traffic. It reports measured problems:
+
+```bash
+lunora insights                       # local dev worker (http://localhost:8787)
+lunora insights --shard channel:demo  # one shard instead of the root
+lunora insights --limit 25 --format json
+LUNORA_ADMIN_TOKEN=… lunora insights --prod --url https://app.example.com
+```
 
-- Scale is small, traffic is modest, and there is no measured problem — prefer
-  simpler code. Do not introduce sharding, digests, or splits on speculation.
+It ranks per-function **write-conflict hot-spots** (go to Write conflicts),
+**error rates**, and **latency outliers** (usually Read amplification). The
+Studio Issues panel and `lunora logs` give error detail once you know where to
+look.
 
-## Core Workflow
+### Static: `lunora advisor` and `@lunora/advisor`
 
-1. **Scope** one concrete user flow with a clear entry and exit.
-2. **Trace** every `ctx.db` read and write in that flow.
-3. **Route** to the matching problem class below.
-4. **Fix siblings** consistently — every function touching the same table should
-   read it the same indexed way.
-5. **Verify** behavior is unchanged and the advisor finding clears.
+No traffic needed. The Studio **Advisors** tab shows the same findings live in
+dev.
 
-## Signal Gathering
+```bash
+lunora advisor                         # score + write lunora.advisor.map.json
+lunora advisor --entry messages#send   # one procedure (file#exportName)
+lunora advisor --baseline              # CI: fail on regression vs the committed map
+```
 
-Start with the static advisors — they need no traffic:
+Lint ids are snake_case. The performance-relevant ones:
 
-- **Lunora Studio → Advisors tab** surfaces `@lunora/advisor` findings live in
-  dev.
-- `@lunora/advisor` runs static lints over `defineSchema` + discovered query
-  reads / insert writes. Relevant performance/schema rules:
-    - `filter-without-index` — a query filters a table with no covering index.
-    - `unindexed-foreign-key` — a relation/FK column has no index.
-    - `duplicate-index` / `empty-index` — wasted or malformed indexes.
-    - `index-references-unknown-field`, `relation-references-unknown-field`,
-      `relation-references-unknown-table` — broken index/relation definitions.
-    - `table-without-insert` — a table is read but never written (often a
-      schema/typo signal).
+| Lint                              | Meaning                                                                      |
+| --------------------------------- | ---------------------------------------------------------------------------- |
+| `filter_without_index`            | A query filters a table with no covering index.                              |
+| `unbounded_collect`               | `.collect()` with no index or filter loads every row (and re-sends it live). |
+| `filter_on_primary_key`           | Filtering on `_id` scans; use `ctx.db.get(id)`.                              |
+| `unindexed_foreign_key`           | An FK column has no index.                                                   |
+| `unindexed_relation_target`       | A `many` relation's target FK has no index, so `with:` reads scan.           |
+| `duplicate_index` / `empty_index` | Wasted or malformed index.                                                   |
+| `index_utilization` (runtime)     | A declared index nothing uses, or a hot table read with none.                |
+| `hot_shard` (runtime)             | One shard takes a disproportionate share of a sharded function's load.       |
+| `fan_out_breadth` (runtime)       | A shard set so wide a cross-shard read nears the subrequest limit.           |
 
-## Problem Class: Read Amplification (the common one)
+## Read amplification (the common one)
 
-**Symptom:** a query reads the whole table to return a few rows;
-`filter-without-index` fires.
+**Symptom:** a query reads the whole table to return a few rows; a latency
+outlier in `insights`, or `filter_without_index` / `unbounded_collect`.
 
-**Fix:** read through an index, not `.filter()`. Declare the index on the table
-and use `.withIndex()` with an equality/range on the leading columns.
+**Fix:** declare an index and read through it with `.withIndex()`:
 
 ```ts
-// Bad — scans every row, then filters in memory.
+// (auth / org membership check on `orgId` elided — see lunora-functions)
+// Scans every row, then filters in memory.
 const mine = (await ctx.db.query("documents").collect()).filter((d) => d.orgId === orgId);
 
-// Good — declare the index…
+// schema.ts
 documents: defineTable({ orgId: v.string(), createdAt: v.number() /* … */ }).index("by_org_created", ["orgId", "createdAt"]);
 
-// …and read through it.
+// Reads only the org's range.
 const mine = await ctx.db
     .query("documents")
     .withIndex("by_org_created", (q) => q.eq("orgId", orgId))
     .collect();
 ```
 
-Index columns are ordered: put equality columns first, then the range/sort
-column. Fix every sibling query on the table the same way.
+Put equality columns first, then the range/sort column. For a large range,
+paginate instead of `.collect()`.
 
-## Problem Class: Write Conflicts (OCC)
+## Relation traversal cost
 
-**Symptom:** mutations on hot rows retry or fail under concurrency. ShardDO uses
-optimistic concurrency control — concurrent writes to the same DO that touch
-overlapping state conflict and retry.
+**Symptom:** a query using `ctx.db.related` is slow, or one page reads far more
+rows than it returns.
 
-**Fixes, in order of preference:**
+A traversal is one batched `findMany` per edge, per hop, and the frontier grows
+multiplicatively. The options are bounded, and an out-of-range value is rejected
+with `BAD_REQUEST` instead of being clamped, so a throwing traversal means the
+request is too big, not that there's a bug to route around.
 
-1. **Narrow the write.** Patch only the fields that changed; avoid read-modify-
-   write over rows another mutation also touches.
-2. **Partition with `.shardBy(key)`.** Move per-user / per-tenant / per-room
-   state into its own DO so writes for different keys never contend. This is the
-   primary horizontal-scale lever — most write contention is a single-DO
-   hotspot.
-3. **Avoid unbounded counters/aggregates in the hot path.** Accumulate in a
-   sharded/append shape and fold lazily rather than serializing every writer
-   through one row.
+| Option        | Default | Cap      | Why                                                           |
+| ------------- | ------- | -------- | ------------------------------------------------------------- |
+| `depth`       | `1`     | `4`      | Past 4 hops, request time is the limit, not `limit`.          |
+| `limit`       | `50`    | `200`    | Nodes per page.                                               |
+| cursor offset | `0`     | `10 000` | The offset is re-walked: each hop reads `N + limit + 1` rows. |
 
-## Problem Class: Subscription Cost
+Fixes, in order:
 
-**Symptom:** a `useQuery` re-runs and re-pushes to many clients on unrelated
-writes.
+1. **Narrow `edges`** to the `"<table>.<column>"` edges the feature needs. One
+   unrelated `v.id(...)` column can double the frontier per hop.
+2. **Set `direction`** to `"in"` or `"out"` instead of the default `"both"`.
+3. **Lower `depth` before raising `limit`.** Hops multiply; a page is linear.
+4. **Index the foreign keys.** Every inward hop is a `WHERE fk IN (…)` read
+   (`unindexed_foreign_key`).
+5. **Stop deep-paging.** Needing an offset past 10 000 means the traversal is
+   the wrong tool; narrow the walk.
+
+## Write conflicts (OCC)
 
-**Fixes:**
+**Symptom:** mutations on hot rows fail under concurrency, or a function tops
+the write-conflict list in `insights`.
 
-- **Scope query args tightly** so a subscription only depends on the rows it
-  renders — a query keyed by `orgId` should not re-run for another org's write.
-- **Read through indexes** (above) so the reactive dependency is the narrow
-  index range, not the whole table.
-- ShardDO subscriptions are **hibernated WebSockets** — idle connections cost
-  nothing; the lever is _how many rows each live query depends on_, not raw
-  connection count.
+ShardDO uses optimistic concurrency. When a concurrent write commits first, the
+mutation throws `ConflictError` (`code: "CONFLICT"`, HTTP 409). The runtime does
+not retry it; a mutation body runs at most once per call, and the caller
+decides what to do. On the client, check with `isConflictError(error)` from
+`@lunora/client` (also exported via `lunorash/client`).
 
-## Problem Class: Cross-Region Reads
+Not every 409 is contention. On the server, `ConflictError.kind` is `"occ"` for
+real write contention, and `"unique"`, `"restrict"` or `"trigger"` for
+constraint and guard failures. The client only sees `code: "CONFLICT"` (`kind`
+is not part of the error envelope), so it cannot tell them apart. `insights` counts only `occ` as a write conflict, so fix a `unique`
+409 in the data or the insert logic, not by sharding.
 
-**Symptom:** read-heavy, rarely-written data is slow for far-away users.
+Fixes, in order:
+
+1. **Narrow the write.** Patch only changed fields; avoid read-modify-write over
+   rows another mutation also writes.
+2. **Partition with `.shardBy(key)`.** Per-user / per-tenant / per-room state
+   in its own Durable Object never contends across keys. This is the main
+   horizontal-scale lever, since most contention is one hot DO.
+3. **Take counters and aggregates off the hot row.** Append and fold lazily
+   instead of serializing every writer through one row.
+
+If `hot_shard` fires on an already-sharded table, the key is skewed: re-shard on
+a higher-cardinality key or split the hot entity's state.
+
+## Subscription cost
+
+**Symptom:** a `useQuery` re-runs and re-pushes to many clients on unrelated
+writes.
 
-**Fix:** chain `.global()` on the table to replicate it to D1 for low-latency
-cross-region reads (with read-your-writes via the Sessions API). Reserve it for
-read-mostly tables — `.global()` adds the D1 migration flow (see the
-`lunora-migration-helper` skill) and write-path cost.
+- Scope query args tightly so a live query depends only on the rows it renders;
+  a query keyed by `orgId` shouldn't re-run on another org's write.
+- Read through indexes so the dependency is the index range, not the table.
+- Idle WebSockets are hibernated and cost nothing. The lever is how many rows
+  each live query depends on, not the connection count.
 
-### `.shardBy(key)` vs `.global()` — choose one per table
+## Cross-region reads
 
-- `.shardBy(key)`: partitions a table across Durable Objects by key — scales
-  _writes_ (e.g. messages per room). Reads are per-shard.
-- `.global()`: replicates a table to D1 — scales _cross-region reads_ with
-  read-your-writes (e.g. a mostly-read catalog).
-- They are not combined on the same table; the default (neither) is a single
-  root-scoped ShardDO.
+**Symptom:** read-heavy, rarely-written data is slow for distant users.
 
-## Guardrails
+**Fix:** `.global()` replicates the table to D1 for low-latency reads from the
+edge, with read-your-writes via D1 sessions. Use it only for read-mostly tables:
+it adds write-path cost and its own DDL flow (`lunora-migration-helper`). If the
+dataset outgrows D1, `.global({ backend: "hyperdrive" })` serves the same
+reactive contract from Postgres/MySQL (`lunora-setup-hyperdrive-global`).
 
-- Prefer simpler code when scale is small or the signal is weak.
-- Do not recommend structural changes (digest tables, document splitting,
-  sharding) without a measured signal or a known hot path.
-- When you change how one function reads/writes a table, change its siblings to
-  match — half-migrated access patterns are their own bug.
+`.shardBy(key)` scales writes (rows in one DO per key); `.global()` scales
+cross-region reads. A table can't have both; the default is the single root
+ShardDO. Switching an existing table's mode moves its rows, which is an
+export/import, not a schema edit (see `lunora-migration-helper`).
 
 ## Checklist
 
-- [ ] Scoped one concrete flow; traced every `ctx.db` read/write.
-- [ ] Checked the Studio Advisors tab / `@lunora/advisor` findings.
-- [ ] Read amplification: replaced `.filter()` with an indexed `.withIndex()`.
-- [ ] Write conflicts: narrowed writes and/or partitioned with `.shardBy(key)`.
-- [ ] Subscription cost: scoped query args so live queries depend on few rows.
-- [ ] Cross-region: applied `.global()` only to read-mostly tables.
-- [ ] Fixed sibling functions consistently; verified behavior unchanged.
+- [ ] Scoped one flow; traced every `ctx.db` read/write.
+- [ ] Checked `lunora insights` (if there's traffic) and `lunora advisor`.
+- [ ] Scans replaced with `.withIndex()` reads; FKs indexed.
+- [ ] Traversals narrowed with `edges` / `direction` before touching `depth`.
+- [ ] Write conflicts confirmed as `occ`, then writes narrowed and/or `.shardBy(key)` applied.
+- [ ] Live queries depend on few rows; `.global()` only on read-mostly tables.
+- [ ] Siblings fixed consistently; behaviour unchanged; finding cleared.