@codyswann/lisa 2.195.3 → 2.195.6

package/plugins/lisa-harper-fabric/.codex-plugin/skills/harper-caching/SKILL.md

@@ -0,0 +1,265 @@
+---
+name: harper-caching
+description: "implementing Harper…"
+---
+
+# Harper Caching
+
+## Overview
+
+Harper can use a local table as a durable cache for an external source. The table
+stores records, enforces schema/index behavior, exposes normal REST/GraphQL
+routes, and calls the source Resource only when a record is missing, expired, or
+explicitly invalidated.
+
+Use a caching table when the cached data should survive process restarts,
+participate in Harper queries, or stay coherent across Fabric nodes. Do not keep
+API responses in a module-level `Map` unless the data is deliberately
+process-local, disposable, and not part of the application data model.
+
+Cross-check the table declaration in [[harper-schema-graphql]] and the Resource
+method conventions in [[harper-resources]] before editing. If cached records are
+queried by filters, sort keys, or relationships, also align indexes and query
+shape with [[harper-rest-queries]].
+
+## Declare the cache table
+
+Declare a normal `@table` and expose it with `@export` when callers should reach
+the cache over REST or GraphQL. Use the `expiration` argument on `@table` for the
+default time-to-live, in seconds:
+
+```graphql
+type WeatherSnapshot @table(expiration: 300) @export(name: "weather") {
+  locationId: String @primaryKey
+  city: String @indexed
+  temperatureF: Float
+  conditions: String
+  sourceUpdatedAt: Long
+  fetchedAt: Long
+}
+```
+
+Guidance:
+
+- Keep the primary key stable and derived from the upstream identity. For an
+  external REST API, a normalized URL slug, vendor id, or compound key string is
+  usually better than an auto-generated id.
+- Index attributes used by cache reads (`city`, `tenantId`, `category`, `status`)
+  so callers do not fetch broad collections and filter in JavaScript.
+- Store upstream freshness metadata such as `sourceUpdatedAt`, `etag`, or
+  `fetchedAt` when verification, debugging, or conditional revalidation needs it.
+- Keep TTL at the table level unless the upstream response controls freshness per
+  record. Per-record expiration belongs in the source Resource context.
+
+## Wire `sourcedFrom`
+
+Implement a source Resource that fetches the upstream record, then attach it to
+the table with `sourcedFrom`. In Lisa template projects, write TypeScript under
+`src/` and rebuild the generated Harper assets; do not edit `resources.js` by
+hand.
+
+```javascript
+export class WeatherApiSource extends Resource {
+  static loadAsInstance = false;
+
+  async get(target) {
+    const id = target.id;
+    const response = await fetch(
+      `https://api.example.com/weather/${encodeURIComponent(id)}`,
+      {
+        headers: { Accept: 'application/json' },
+      },
+    );
+
+    if (response.status === 404) {
+      const error = new Error(`Weather location not found: ${id}`);
+      error.statusCode = 404;
+      throw error;
+    }
+
+    if (!response.ok) {
+      const error = new Error(`Weather upstream failed: ${response.status}`);
+      error.statusCode = 502;
+      throw error;
+    }
+
+    const data = await response.json();
+
+    return {
+      locationId: id,
+      city: data.city,
+      temperatureF: data.temperatureF,
+      conditions: data.conditions,
+      sourceUpdatedAt: Date.parse(data.updatedAt),
+      fetchedAt: Date.now(),
+    };
+  }
+}
+
+tables.WeatherSnapshot.sourcedFrom(WeatherApiSource);
+```
+
+When a requested record is missing or stale, Harper calls the source `get()` and
+caches the returned record in the local table. Concurrent requests for the same
+missing or stale record share the same upstream load, which avoids a cache
+stampede for a single key.
+
+## TTL and eviction
+
+Use `expiration` for the TTL: after that many seconds, the cached entry is stale
+and the next read reloads it from the source. If a project cannot express the
+default in schema, `sourcedFrom` can also receive options:
+
+```javascript
+tables.WeatherSnapshot.sourcedFrom(WeatherApiSource, {
+  expiration: 300,
+  eviction: 3600,
+  scanInterval: 60,
+});
+```
+
+Option meanings:
+
+| Option | Use |
+| --- | --- |
+| `expiration` | Default TTL in seconds before the cached record is stale. |
+| `eviction` | Seconds after expiration before Harper may physically remove the record. |
+| `scanInterval` | Seconds between scans for expired records. |
+
+Prefer schema directives for stable application behavior because the data model
+and default freshness policy stay together. Use `sourcedFrom` options when the
+cache attachment is intentionally dynamic or the downstream Harper version lacks
+the needed schema directive.
+
+## Conditional revalidation
+
+When the upstream provides `ETag`, `Last-Modified`, or `Cache-Control`, pass that
+freshness model through the source Resource instead of overwriting good cached
+data with every revalidation.
+
+```javascript
+export class WeatherApiSource extends Resource {
+  static loadAsInstance = false;
+
+  async get(target) {
+    const context = this.getContext();
+    const headers = new Headers({ Accept: 'application/json' });
+
+    if (context.replacingVersion) {
+      headers.set(
+        'If-Modified-Since',
+        new Date(context.replacingVersion).toUTCString(),
+      );
+    }
+
+    const response = await fetch(
+      `https://api.example.com/weather/${encodeURIComponent(target.id)}`,
+      { headers },
+    );
+
+    if (response.status === 304) {
+      return context.replacingRecord;
+    }
+
+    const maxAge = response.headers
+      .get('Cache-Control')
+      ?.match(/max-age=(\d+)/)?.[1];
+
+    if (maxAge) {
+      context.expiresAt = Date.now() + Number(maxAge) * 1000;
+    }
+
+    context.lastModified = response.headers.get('Last-Modified');
+    return response.json();
+  }
+}
+```
+
+Use `allowStaleWhileRevalidate(entry, id)` on the cached table when stale data is
+acceptable during refresh:
+
+```javascript
+export class WeatherSnapshot extends tables.WeatherSnapshot {
+  static loadAsInstance = false;
+
+  allowStaleWhileRevalidate(entry, id) {
+    return Date.now() - entry.expiresAt < 60_000;
+  }
+}
+```
+
+Return `false` or omit the method when callers must wait for a fresh value.
+
+## Explicit invalidation
+
+Use invalidation when an admin action, webhook, scheduled refresh, or upstream
+write makes the cached record stale before its TTL.
+
+```javascript
+await tables.WeatherSnapshot.invalidate('nyc');
+```
+
+The next `get()` for that id reloads from the source. If the project Harper
+version or resource shape does not expose `invalidate()`, use an explicit delete
+or overwrite path and document the behavior:
+
+```javascript
+await tables.WeatherSnapshot.delete('nyc');
+await tables.WeatherSnapshot.get('nyc');
+```
+
+For write-through caches, implement `put`, `patch`, or `delete` on the source
+Resource only when those operations should update the upstream system. Otherwise,
+reject writes or keep cache mutation behind an operator-only Resource so clients
+do not accidentally diverge from the source of truth.
+
+## Fabric coherence
+
+In a Fabric deployment, treat every node as capable of serving a read while cache
+state is replicating:
+
+- Keep `get()` deterministic and idempotent. A second node may reload the same
+  stale key.
+- Do not store request-local data, credentials, or tenant state in module-level
+  variables. Read identity from Resource context and pass context through when
+  calling other resources.
+- Pick TTLs that tolerate replication delay. Very short TTLs can turn a cache
+  into repeated upstream traffic across nodes.
+- After invalidation, verify from the route or node that matters for the caller.
+  Cross-node visibility may lag until replication catches up.
+- Use Harper storage reclamation settings for disk-pressure eviction tuning, not
+  as a substitute for application freshness rules.
+
+## Local verification recipe
+
+Prove both cache fill and cache hit behavior against a running Harper app:
+
+1. Add a temporary upstream counter, fixture server log, or test-only header that
+   shows how many times the source Resource fetched the upstream id.
+2. Build and boot the app (`bun run build`, then the project Harper dev command).
+3. Request the same id twice:
+
+```bash
+curl -sS http://localhost:9926/weather/nyc | jq .
+curl -sS http://localhost:9926/weather/nyc | jq .
+```
+
+4. Confirm the upstream was called once, the second response came from the table,
+   and the record contains `fetchedAt` or equivalent freshness evidence.
+5. Invalidate or delete the id, request it again, and confirm the upstream count
+   increments:
+
+```bash
+curl -sS -X DELETE http://localhost:9926/weather/nyc
+curl -sS http://localhost:9926/weather/nyc | jq .
+```
+
+6. If TTL behavior matters, lower `expiration` in a local-only test fixture,
+   wait past the TTL, and verify either blocking refresh or stale-while-revalidate
+   behavior matches the Resource implementation.
+
+## Sources
+
+- [Harper v4 Resource API](https://docs.harperdb.io/reference/v4/resources/resource-api)
+- [Harper v5 Resources overview](https://docs.harperdb.io/reference/v5/resources/overview)
+- [Harper storage tuning](https://docs.harperdb.io/reference/v5/database/storage-tuning)

package/plugins/lisa-harper-fabric/.codex-plugin/skills/harper-caching/agents/openai.yaml

@@ -0,0 +1,4 @@
+display_name: "Harper Caching"
+short_description: "implementing Harper…"
+default_prompt:
+  - "Use $harper-caching: implementing Harper…."

package/plugins/lisa-harper-fabric/.codex-plugin/skills/harper-component-model/SKILL.md

@@ -0,0 +1,87 @@
+---
+name: harper-component-model
+description: "reasoning about how a Harper…"
+---
+
+# Harper Component Model
+
+## Overview
+
+Harper (formerly HarperDB; the product and company now at harper.fast) is an
+open-source Node.js platform that fuses **database, cache, application logic, and
+messaging into a single in-memory process**. **Fabric** is Harper's distributed
+deploy network: you develop locally, then deploy the same component to Fabric.
+
+Everything you build is a **component**. Understanding the component hierarchy is
+the prerequisite for every other Harper decision — config, resources, schema, and
+deploy all hang off it.
+
+## The three tiers
+
+Harper organizes functionality into three tiers, top to bottom:
+
+1. **Applications** — the user-facing product. An application *is* a type of
+   component. It implements business logic (REST/GraphQL endpoints, web UI,
+   real-time) by depending on extensions/plugins. This is what this project is.
+2. **Plugins** (v5) / **Extensions** (deprecated) — the building blocks an
+   application depends on. A plugin exports a single `handleApplication(scope)`
+   method and always runs on worker threads. The deprecated Extension API used
+   `start`, `handleFile`, `handleDirectory`, and `setupDirectory` instead.
+   `handleApplication()` cannot coexist with Extension API methods — defining
+   both throws. Prefer the Plugin API for any custom building block.
+3. **Core services** — the high-performance database, networking middleware, and
+   the component manager. You configure these; you don't reimplement them.
+
+> An *application* is the component you ship. *Extensions/plugins* are the
+> capabilities it consumes. Built-in extensions (`graphqlSchema`, `jsResource`,
+> `rest`, `static`, …) are provided by core; you only *enable* them in
+> `config.yaml`. See [[harper-config-yaml]].
+
+## Built-in extensions an application typically uses
+
+These ship with Harper and are enabled (not installed) via `config.yaml`:
+
+- `graphqlSchema` — define database tables/types from GraphQL schema files. See [[harper-schema-graphql]].
+- `jsResource` — load custom JavaScript resources (`resources.js`). See [[harper-resources]].
+- `rest` — auto-generate REST endpoints for exported resources/tables.
+- `static` — serve static files (the `web/**` directory) over HTTP.
+- `roles` — role-based access control from `roles.yaml`.
+- `loadEnv` — load environment variables from `.env`.
+- `dataLoader` — seed Harper tables from JSON/YAML.
+- `fastifyRoutes` — custom Fastify route handlers.
+
+## Where things live in this project
+
+This project wraps Harper's native model with a fixed layout under `harper-app/`:
+
+| Path | Role | Source or generated |
+| --- | --- | --- |
+| `harper-app/config.yaml` | Component config — which extensions are active | **Source** |
+| `harper-app/schema.graphql` | Table/type definitions | **Source** |
+| `src/**` (TypeScript) | Resources, browser modules, shared libs, scripts | **Source** |
+| `harper-app/resources.js` | Loaded by `jsResource` | **Generated** — never edit; build from TS |
+| `harper-app/web/**` | Served by `static` | **Generated** — never edit; build from TS |
+
+The TypeScript under `src/` is the source of truth. `resources.js` and `web/**`
+are deploy artifacts produced by `bun run build`. Never hand-edit them — change
+the matching TypeScript and rebuild. See [[harper-build-and-deploy]].
+
+## Decision guide
+
+- **Adding a backend behavior?** It's almost always a *resource* (custom logic) or
+  a *schema* change (new table/field), enabled through `config.yaml`. Don't ship a
+  client-side workaround for missing backend behavior — make the Harper change.
+- **Adding a reusable building block / npm-publishable capability?** That's a
+  *plugin* (`pluginModule`), not application code.
+- **Adding static UI?** It belongs in `web/**` (generated from `src/` UI code), served
+  by the `static` extension.
+- **Not sure if it deploys?** A deployable Harper app must keep `config.yaml`,
+  `schema.graphql`, `resources.js`, and `web/**` at the component root that Fabric
+  packages. If your change touches that surface, build before packaging.
+
+## Sources
+
+- [Components overview](https://docs.harperdb.io/reference/v5/components/overview)
+- [Plugin API](https://docs.harperdb.io/reference/v5/components/plugin-api)
+- [Applications](https://docs.harperdb.io/docs/developers/applications)
+- [Harper platform](https://www.harper.fast/)

package/plugins/lisa-harper-fabric/.codex-plugin/skills/harper-component-model/agents/openai.yaml

@@ -0,0 +1,4 @@
+display_name: "Harper Component Model"
+short_description: "reasoning about how a Harper…"
+default_prompt:
+  - "Use $harper-component-model: reasoning about how a Harper…."

package/plugins/lisa-harper-fabric/.codex-plugin/skills/harper-config-yaml/SKILL.md

@@ -0,0 +1,283 @@
+---
+name: harper-config-yaml
+description: "creating or editing a Harper…"
+---
+
+# Harper config.yaml
+
+## Overview
+
+A Harper component is configured by a single `config.yaml` at the component root
+(`harper-app/config.yaml` in this project). It declares **which extensions are
+active** and which files each extension reads. Extensions are *enabled* here, not
+installed — the built-ins ship with Harper.
+
+The structure is a map of extension name to its options:
+
+```yaml
+extensionName:
+  option-1: value
+  option-2: value
+```
+
+## ⚠️ The no-merge footgun
+
+If a component has **no** `config.yaml`, Harper applies this default automatically:
+
+```yaml
+rest: true
+graphqlSchema:
+  files: '*.graphql'
+roles:
+  files: 'roles.yaml'
+jsResource:
+  files: 'resources.js'
+fastifyRoutes:
+  files: 'routes/*.js'
+  urlPath: '.'
+static:
+  files: 'web/**'
+```
+
+The moment you add a **custom `config.yaml`, it replaces this default entirely —
+Harper does not merge your file with the defaults.** If you write a `config.yaml`
+that only enables `static`, you have silently turned off `rest`, `graphqlSchema`,
+`jsResource`, and `roles`. This is the most common Harper config mistake.
+
+**Rule:** when you add or edit `config.yaml`, re-declare every extension the app
+actually needs — start from the default block above and add to it. Do not assume
+unmentioned extensions stay on.
+
+## Built-in extensions
+
+| Key | Purpose | Common options |
+| --- | --- | --- |
+| `rest` | Auto REST endpoints and resource WebSocket subscriptions | `true`, or an object with options such as `webSocket` |
+| `graphqlSchema` | Define tables/types from GraphQL files | `files: '*.graphql'` — see [[harper-schema-graphql]] |
+| `jsResource` | Load custom JS resources | `files: 'resources.js'` — see [[harper-resources]] |
+| `static` | Serve static files over HTTP | `files: 'web/**'`, `urlPath` |
+| `roles` | Role-based access control | `files: 'roles.yaml'` |
+| `loadEnv` | Load env vars from `.env` | `files` |
+| `dataLoader` | Seed tables from JSON/YAML | `files` |
+| `fastifyRoutes` | Custom Fastify routes | `files: 'routes/*.js'`, `urlPath` |
+
+For real-time work, component `config.yaml` keeps `rest`, `graphqlSchema`, and
+`jsResource` enabled so exported resources can be addressed by HTTP/WebSocket and
+MQTT topic paths. Broker ports and MQTT authentication live in the root
+`harper-config.yaml`, not the component file. See [[harper-realtime]].
+
+## `dataLoader`: seed data
+
+Use `dataLoader` for versioned seed/reference records that should exist whenever
+the component is deployed. Define table shape first with `graphqlSchema`, then
+point `dataLoader.files` at one or more JSON/YAML files:
+
+```yaml
+graphqlSchema:
+  files: 'schema.graphql'
+dataLoader:
+  files:
+    - 'data/roles.yaml'
+    - 'data/reference/*.json'
+```
+
+Each data file targets exactly one table and has `database`, `table`, and
+`records` keys:
+
+```yaml
+database: app
+table: Role
+records:
+  - id: admin
+    name: Administrator
+    permissions:
+      - users:read
+      - users:write
+  - id: viewer
+    name: Viewer
+    permissions:
+      - users:read
+```
+
+Harper runs the loader on full system starts and component deployments. It is
+safe to re-run when files are idempotent: new records are inserted, unchanged
+records are skipped, and records are updated from the file only when the tracked
+file content changed. User-created records and user edits made after an initial
+load are preserved; changed data-loaded records are patched instead of blindly
+replaced.
+
+Choose `dataLoader` for small, source-controlled reference/configuration data
+that should ship with the component. Use a REST/Operations API script or job for
+large imports, environment-specific backfills, or one-off migrations where retry
+scope and operator approval matter.
+
+Verify locally:
+
+```bash
+harper dev harper-app
+curl -s http://localhost:9926/app/Role/admin
+harper dev harper-app # restart/redeploy and confirm the seed did not duplicate
+```
+
+## `fastifyRoutes`: custom HTTP routes
+
+Prefer `jsResource` plus `rest` for normal CRUD/action APIs. Use `fastifyRoutes`
+only when the route shape does not fit the Resource model: webhooks, custom
+serialization, unusual path matching, or a compatibility endpoint.
+
+```yaml
+rest: true
+graphqlSchema:
+  files: 'schema.graphql'
+jsResource:
+  files: 'resources.js'
+fastifyRoutes:
+  files: 'routes/*.js'
+  urlPath: 'hooks'
+```
+
+Route modules default-export an async function that receives the Fastify server
+and Harper helpers:
+
+```js
+export default async (server, { hdbCore, logger }) => {
+  server.route({
+    method: 'POST',
+    url: '/payment/:provider',
+    preValidation: hdbCore.preValidation,
+    handler: async (request, reply) => {
+      logger.debug(`payment webhook ${request.params.provider}`);
+      request.body = {
+        operation: 'insert',
+        schema: 'app',
+        table: 'WebhookEvent',
+        records: [
+          {
+            id: request.headers['x-event-id'],
+            provider: request.params.provider,
+            payload: request.body,
+          },
+        ],
+      };
+      const result = await hdbCore.request(request);
+      return { ok: true, result };
+    },
+  });
+};
+```
+
+Use Fastify's `request.params`, `request.query`, `request.body`, and `request.headers`
+for route inputs. Keep auth explicit: `hdbCore.request` should be paired with
+`hdbCore.preValidation` so Harper authenticates the request. Avoid
+`requestWithoutAuthentication` unless the route has its own signature/JWT check
+and all user-provided values are bound or escaped; never build SQL strings by
+interpolating params/body values.
+
+Verify locally:
+
+```bash
+harper dev harper-app
+curl -i -X POST http://localhost:9926/app/hooks/payment/stripe \
+  -H 'Authorization: Basic ...' \
+  -H 'Content-Type: application/json' \
+  -H 'x-event-id: evt_123' \
+  --data '{"status":"paid"}'
+```
+
+## `static`: serve web assets and SPAs
+
+Use `static` to serve generated browser output or other immutable assets from
+the component. In Lisa Harper Fabric projects, `harper-app/web/**` is generated
+by the project build; edit the source UI under `src/`, not the deployed files.
+
+```yaml
+static:
+  files: 'web/**'
+  urlPath: '.'
+  index: true
+```
+
+`files` selects what is served. `urlPath` mounts those files under a URL prefix:
+`urlPath: 'app'` makes `web/index.html` available at `/app/index.html`; the
+default application path still includes the Harper project/component prefix. Use
+`index: true` to serve `index.html` for directory requests, and `extensions:
+['html']` when clean URLs should resolve to `.html` files.
+
+For client-side-routed SPAs, return the app shell for unmatched asset paths:
+
+```yaml
+static:
+  files: 'web/**'
+  urlPath: '.'
+  index: true
+  fallthrough: false
+  notFound:
+    file: 'web/index.html'
+    statusCode: 200
+```
+
+That fallback is for browser routes such as `/reports/weekly`; it should not hide
+missing API endpoints or broken asset names. Keep API routes under a clear
+prefix, and check that hashed JS/CSS assets still return their actual files.
+
+Harper's documented `static` config controls path matching and not-found
+behavior, not custom cache policy. Treat MIME type and cache headers as runtime
+behavior to verify with `curl -I`; if the app needs precise cache headers,
+front it with an edge/proxy policy or a custom route designed for that asset
+surface.
+
+Verify locally:
+
+```bash
+harper dev harper-app
+curl -I http://localhost:9926/app/
+curl -I http://localhost:9926/app/assets/index.js
+curl -I http://localhost:9926/app/client-side-route
+```
+
+## External components and custom plugins
+
+A component you depend on from npm needs a `package:` directive matching a
+`package.json` dependency:
+
+```yaml
+'@harperdb/nextjs':
+  package: '@harperdb/nextjs'
+  files: './'
+```
+
+A custom plugin you author is wired with `pluginModule` (point at the built JS, not
+TypeScript source):
+
+```yaml
+pluginModule: ./dist/index.js
+```
+
+The deprecated Extension API used `extensionModule` instead. Prefer `pluginModule`.
+See [[harper-component-model]] for the plugin-vs-extension distinction.
+
+## Project conventions
+
+- `config.yaml` is **source** and lives at `harper-app/config.yaml`. It is part of
+  the deployable surface Fabric packages — keep it at the component root.
+- The files `config.yaml` points to (`resources.js`, `web/**`) are **generated** by
+  `bun run build` from TypeScript under `src/`. Editing `config.yaml` to point at a
+  new file means the build must produce that file. See [[harper-build-and-deploy]].
+- A `config.yaml` change is a deploy-shape change: update the matching project doc
+  (the Fabric runbook) in the same change, and re-run the smoke command.
+- Keep secrets out of `config.yaml`. Use `loadEnv` / environment variables and
+  document *where* secrets live without recording their values.
+
+## Verification
+
+After editing `config.yaml`, confirm the app still boots and the expected surface
+is live: run `harper dev harper-app` (or the project's run command) and check that
+the REST/GraphQL/static endpoints you rely on respond. A config that silently
+dropped an extension often fails only at runtime, not at build time.
+
+## Sources
+
+- [Components overview](https://docs.harperdb.io/reference/v5/components/overview)
+- [Data Loader](https://docs.harperdb.io/reference/v5/database/data-loader)
+- [Fastify Routes](https://docs.harperdb.io/reference/v5/fastify-routes/overview)
+- [Static Files](https://docs.harperdb.io/reference/v5/static-files/overview)

package/plugins/lisa-harper-fabric/.codex-plugin/skills/harper-config-yaml/agents/openai.yaml

@@ -0,0 +1,4 @@
+display_name: "Harper Config Yaml"
+short_description: "creating or editing a Harper…"
+default_prompt:
+  - "Use $harper-config-yaml: creating or editing a Harper…."