spine-framework 0.4.4 → 0.5.0-rc.5

package/.framework/src/instructions/concepts/design-schema.md

@@ -0,0 +1,152 @@
+---
+title: Design Schema
+audience: developers, agents
+kind: concept
+verified_against: f69c37d
+verified_on: 2026-10-29
+source_files:
+  - .framework/src/types/types.ts
+  - .framework/functions/types.ts
+  - .framework/functions/admin-data.ts
+  - .framework/functions/_shared/schema-utils.ts
+  - .framework/functions/_shared/permissions.ts
+  - .framework/functions/_shared/resolve-ids.ts
+  - .framework/src/components/app-shell/GenericListPage.tsx
+  - .framework/src/components/app-shell/GenericDetailPage.tsx
+  - .framework/src/hooks/useListSchema.ts
+  - .framework/migrations/058_account_data_field_declarations.sql
+  - .framework/migrations/060_account_person_record_permissions.sql
+---
+
+# Design Schema
+
+`design_schema` is the JSON contract on a `public.types` row that declares
+everything the platform needs to store, validate, permission, and render one
+kind of runtime record — without code. It is the single most important
+structure in Spine: get it right and permissions, validation, and the generic
+UI all work for free; get it wrong or leave it out and records are created
+unpermissionable and unrenderable.
+
+The TypeScript shape is `DesignSchema` in `.framework/src/types/types.ts`:
+`fields`, `views`, `record_permissions`, plus optional `scope` and
+`functionality`.
+
+## Where it lives and how it propagates
+
+- The **type definition** holds the canonical copy in
+  `public.types.design_schema` (created/updated only via
+  `.framework/functions/types.ts`, which requires `system_admin`).
+- **Every record created through `admin-data` is stamped** with the type's
+  `design_schema` and `validation_schema` at insert time
+  (`.framework/functions/admin-data.ts`, `create`). The stamped copy on the
+  record — not the type row — is what the permission engine reads at request
+  time (`.framework/functions/_shared/permissions.ts`,
+  `sanitizeFirstSurfaceRecordData` / `validateFirstSurfaceUpdatePermissions`
+  both refuse when the record carries no `record_permissions`).
+- **Editing a type does not restamp existing records.** Migrations that change
+  a type schema re-stamp rows explicitly — see
+  `.framework/migrations/058_account_data_field_declarations.sql` and
+  `.framework/migrations/060_account_person_record_permissions.sql` for the
+  pattern.
+- `validation_schema` is a derived artifact: `generateValidationSchema` in
+  `.framework/functions/_shared/schema-utils.ts` extracts `data_type`,
+  `required`, `validation`, `options`, and `reference_*` from `fields` and runs
+  on every type create and on every update that touches `design_schema`
+  (`.framework/functions/types.ts`). Never hand-edit it — change
+  `design_schema` and let the API regenerate.
+
+## The sections
+
+### `scope`
+
+`"account"` (default when omitted) or `"platform"`. At create time `admin-data`
+reads `designSchema.scope ?? 'account'`; `platform` records are stamped with
+the `spine-system` account's id instead of the caller's account. Child records
+created against a parent (via `thread_id`, `target_id`, `parent_id`) also
+inherit the parent's stamped scope. See
+[Scope and Accounts](scope-and-accounts.md).
+
+### `fields`
+
+A flat map of `snake_case` field names to `FieldDefinition`
+(`.framework/src/types/types.ts`): `data_type`, `label`, `required`, plus
+optional `system`, `placeholder`, `validation` (pattern, min/max, lengths,
+`reference_kind`/`reference_type`, `currency_code`), `options` (required for
+`select`/`multiselect`/`radio`), and `permissions` (per-role `read`/`write`
+overrides — see [Permissions](permissions.md)).
+
+The `system` flag controls storage: `system: true` fields are top-level record
+columns; all others live under `record.data.<field>` — see
+[Runtime Records](runtime-records.md).
+
+Known `data_type` values (the authoritative switch is `sanitizeFieldData` in
+`.framework/functions/_shared/schema-utils.ts`): `text`, `textarea`,
+`rich_text`, `email`, `phone`, `url`, `number`, `currency`, `range`, `date`,
+`datetime`, `boolean`, `checkbox`, `select`, `multiselect`, `radio`, `color`,
+`file`, `image`, `json`, `reference`, `address`. Unknown types pass through
+unsanitized — they are not rejected, they are unvalidated.
+
+### `views`
+
+A map of view names to view configs. `default_list` and `default_detail` are
+what the generic renderer consumes:
+
+- `default_list` (`type: "list"`, `display: "table" | "card" | "board"`,
+  `fields` map with `sortable`/`display_type`, `default_sort`, optional
+  `filters`/`stats`) becomes columns in `GenericListPage`
+  (`.framework/src/components/app-shell/GenericListPage.tsx`) — the schema
+  itself resolves via `useListSchema`, which prefers a stamped record's
+  `design_schema` and falls back to the types API.
+- `default_detail` (`type: "detail"`, `sections[]` of `{title, fields}`) drives
+  `GenericDetailPage`'s `SchemaDetailForm`; each section field resolves its
+  `FieldDefinition` and reads `record[field]` or `record.data[field]` by
+  `system` (`.framework/src/components/app-shell/GenericDetailPage.tsx`).
+
+`display_type` values for view fields come from `ViewFieldConfig` in
+`.framework/src/types/types.ts` (`input`, `text`, `badge`, `timestamp`,
+`select`, `date_picker`, `file_upload`, …). An app with `renderer = "generic"`
+is broken without these views; a `custom` app may omit them but loses the
+generic admin/runtime fallback.
+
+### `record_permissions`
+
+`{ "<role_slug>": ["create" | "read" | "update" | "delete", ...] }` — the base
+grant for the record, evaluated per request against the caller's role plus the
+`'all'` wildcard role. This is the contract evaluated in
+`.framework/functions/_shared/permissions.ts`; the full model is in
+[Permissions](permissions.md).
+
+If a type has no `record_permissions`, `admin-data` create merges a permissive
+default (`{ all: ["create","read","update","delete"], fields: {} }`) before
+stamping — an explicit empty object would instead deny everything downstream,
+so always author them deliberately.
+
+### `functionality`
+
+Optional automation bindings the runtime interprets — e.g.
+`functionality.lineage` (parent/child resolution for items, read in
+`admin-data` list/get) and the integration/constraint bindings declared in
+`FunctionalityBindings` (`field_map` sync, `unique`/`conditional_required`/
+`immutable` constraints) in `.framework/src/types/types.ts`. Set it to `null`
+until the automation binding you need is confirmed live.
+
+## What each consumer reads
+
+| Consumer | Reads |
+|---|---|
+| `admin-data` create | `fields`/`record_permissions`/`scope` for stamping, gating, and account resolution |
+| `permissions.ts` | `record_permissions`, `fields.*.permissions` — on the **record's** stamped schema |
+| `schema-utils.ts` | generated `validation_schema` for `sanitizeFieldData` / `formatFieldData` |
+| `GenericListPage` | `views.default_list` (columns, sort, filters, stats) |
+| `GenericDetailPage` | `views.default_detail`, `fields`, `display_field` |
+| `useEntityRecord` | `fields.*.system` to split system vs `data` writes |
+
+## Authoring minimum
+
+Every type needs, at minimum: `fields` (each with `data_type`, `label`,
+`required`), `record_permissions` (at least `system_admin`), and — for generic
+apps — `default_list` + `default_detail`. `slug` must be unique per `kind`
+(enforced by the `types_kind_slug_unique` index in
+`.framework/migrations/015_types_kind_slug_unique.sql`), and `kind` must match
+the runtime table the type governs. The entity page
+[../types.md](../types.md) has minimal and full examples.

package/.framework/src/instructions/concepts/integrations-and-webhooks.md

@@ -0,0 +1,127 @@
+---
+title: "Integrations and webhooks"
+audience: developers, agents
+kind: concept
+verified_against: f69c37d
+verified_on: 2026-10-29
+source_files:
+  - .framework/functions/integrations.ts
+  - .framework/functions/integration-outbound.ts
+  - .framework/functions/integration-routes.ts
+  - .framework/functions/deliveries.ts
+  - .framework/functions/_shared/webhook-registration.ts
+  - .framework/functions/_shared/webhook-registry.ts
+  - .framework/functions/_shared/resolve-credentials.ts
+  - .framework/functions/_shared/retry-deliveries.ts
+  - .framework/functions/_shared/notify.ts
+  - .framework/functions/adapters/generic-webhook-outbound.ts
+  - .framework/migrations/004_core_rpc_and_broker.sql
+---
+
+# Integrations and webhooks
+
+An **integration** (`public.integrations`) is a named bundle of `config` +
+`credentials` that outbound calls and inbound webhooks share. Spine routes both
+directions through `public.webhook_deliveries`, so every attempt is a durable
+row you can list, inspect, and retry — there is no fire-and-forget.
+
+## The integration row
+
+`integrations` (`.framework/functions/integrations.ts`) requires `name`,
+`integration_type`, and `provider` on create; it also carries `app_id`,
+`version`, `config`, `credentials`, `metadata`, and a computed
+`is_configured` flag.
+
+Credentials live in a dedicated `credentials` jsonb column (added by migration
+`004`) shaped `{ <field>: { source: "value" | "env_var", value: string } }`.
+`resolve-credentials` resolves them **server-side only**, inside the functions
+runtime: `source: "env_var"` reads `process.env[value]`; `source: "value"`
+returns the literal string. Every resolution (or failure) emits an
+`integration.credential_*` audit event. Non-system-admin, non-machine API reads
+of an integration row are sanitized to minimal fields — credential values are
+never returned to ordinary users, though they are stored in the database (use
+`env_var` sources for the tightest posture).
+
+## Outbound calls
+
+`integration-outbound` (`.framework/functions/integration-outbound.ts`) is the
+single egress path — `callOpenAI` in `_shared/llm.ts`, the `http_request`,
+`webhook`, `send_email`, and `agent_inference` pipeline stages, and manual
+calls all go through its `execute`:
+
+1. **Resolve the integration**: account-scoped row matching
+   `integration_slug` first, then a system integration (`account_id IS NULL`).
+   Both must be `is_active`.
+2. **Open a delivery row**: inserts `webhook_deliveries` with
+   `direction: 'outbound'`, `status: 'pending'`, `attempt_count: 0`, and
+   `max_attempts` from `action_config.max_attempts` →
+   `integration.config.retry_policy.max_attempts` → default 3.
+3. **Resolve the credential** (`action_config.credential_field`, default
+   `api_token`) — skipped when the entry is absent or an empty
+   `source: 'value'` literal, which means "call without auth"; the adapter
+   then decides whether a key is required — and **pick an adapter**:
+   `action_config.adapter || integration.config.outbound_adapter`. Built-ins:
+   `adapters/openai-outbound`, `adapters/generic-webhook-outbound`,
+   `adapters/resend-outbound`; anything else is dynamically imported as a
+   sibling module exposing `buildRequest`.
+4. **Send**: the adapter builds `{ url, method, headers, body }`; the function
+   fetches it, then updates the delivery to `delivered` (with
+   `response_status`, `response_body`, `duration_ms`) or `pending`/`failed`.
+5. Optionally, `action_config.update_record` stamps
+   `last_outbound_status` / `last_outbound_at` back onto the source item.
+
+## Delivery and retry semantics
+
+Both directions share the same retry math: `computeNextAttempt` is exponential
+backoff — `30s * 2^attempt`, capped at 1 hour, with up to 20% jitter. A failed
+attempt stays `pending` with `next_attempt_at` set until `attempt_count`
+reaches `max_attempts`, when it becomes `failed`.
+
+Retries are driven by `retryPendingDeliveries`
+(`.framework/functions/_shared/retry-deliveries.ts`), which `system-cron` runs
+every tick (see [automation](./automation.md)). It sweeps `webhook_deliveries`
+rows that are `pending`, `attempt_count < 10`, and `next_attempt_at` due (or
+null), up to 100 per pass, and re-invokes `integration-outbound` with
+`delivery_id` under the system principal — the `delivery_id` path reuses the
+existing row instead of inserting a duplicate. You can also retry one delivery
+manually via `POST /api/deliveries?action=retry { id }`
+(`.framework/functions/deliveries.ts`); manual retry is outbound-only. Inbound
+retry is not implemented — inbound failures are counted and logged.
+
+`deliveries` list/get are account-scoped reads over the same table, filterable
+by `direction`, `status`, `integration_id`, `api_key_id`.
+
+## Inbound webhooks
+
+`integration-routes` (`.framework/functions/integration-routes.ts`) is a
+`createPublicHandler` endpoint — it resolves the caller as an **API-key
+machine principal** (see [auth](./auth-and-api-keys.md)):
+
+1. `?slug=` names the integration; the caller's API key must belong to the
+   same account, and keys pinned to `integration_id` or `app_id` can only hit
+   those integrations.
+2. The event name comes from the `x-event-type` header or
+   `integration.config.event_path` (a dot-path into the payload).
+3. `config.handler_name || provider || 'generic-webhook'` is resolved through
+   the **webhook registry** (`_shared/webhook-registry.ts`): the
+   `webhook_handlers` table maps a handler name to a `function_name`, scoped
+   app → account → global fallback. The named module is dynamically imported
+   (`custom/…` paths resolve into `custom/`) and must export `adapt`,
+   `handler`, or a default function.
+4. The adapter returns `{ type_slug, data, title?, status? }` — the route then
+   creates an `items` record through `admin-data` (full permission/type/audit
+   pipeline) and marks the inbound delivery `delivered`. Adapter errors marked
+   `retryable` leave it `pending`.
+
+Handlers are registered with `registerWebhookHandler`
+(`_shared/webhook-registration.ts`), an upsert on `(app_id, account_id,
+name)`; `deregisterWebhookHandler` soft-deletes (`is_active=false`,
+`is_deleted=true`).
+
+## Notifications
+
+`_shared/notify.ts` resolves pipeline `send_notification` recipients: each
+entry is either a person UUID or `role:<slug>`, resolved to active `people`
+rows in the caller's account via the `role_id → roles.slug` join. Anything
+else is skipped with a warning. The resulting ids feed the `notifications`
+insert in the pipeline runner.

package/.framework/src/instructions/concepts/observability.md

@@ -0,0 +1,116 @@
+---
+title: "Observability: logs, deliveries, executions"
+audience: developers, agents
+kind: concept
+verified_against: f69c37d
+verified_on: 2026-10-29
+source_files:
+  - .framework/functions/logs.ts
+  - .framework/functions/observability.ts
+  - .framework/functions/deliveries.ts
+  - .framework/functions/pipeline-executions.ts
+  - .framework/functions/item-progress.ts
+  - .framework/functions/_shared/logging.ts
+  - .framework/functions/_shared/tracing.ts
+  - .framework/functions/_shared/audit.ts
+  - .framework/migrations/069_log_stats_window.sql
+---
+
+# Observability: logs, deliveries, executions
+
+Everything an operator needs lands in a small set of queryable tables. There is
+no external telemetry service: `public.logs` is the unified sink, and the
+`logs`, `observability`, `deliveries`, and `pipeline-executions` functions are
+the read APIs over it.
+
+## What gets written, and where
+
+`public.logs` holds three overlapping kinds of rows:
+
+- **Audit/log events** — `emitLog`, `emitAudit`, and `emitActivity` in
+  `_shared/audit.ts` insert business events (`trigger.fired`,
+  `pipeline.completed`, `integration.outbound.failed_http`,
+  `api_key.created`, `registration.completed`, …) with actor, target, and
+  details.
+- **Spans** — `emitSpan` writes trace spans (`span_name`, `trace_id`,
+  `span_id`, `parent_span_id`, `duration_ms`, `outcome`). `_shared/tracing.ts`
+  implements W3C `traceparent` parsing/formatting, and `traceDb` wraps the
+  request's DB client so every query emits a `db.<operation>` span — a single
+  `trace_id` stitches the request, its queries, and the pipeline stages it
+  caused.
+- **Operation logs** — `logAdminDataOperation` in `_shared/logging.ts` writes
+  admin-data CRUD records, gated by resolved log levels.
+
+Log **levels** are `'off' | 'timing' | 'warnings' | 'errors' | 'all'`, resolved
+per request from `system_settings` rows (`key='logging'`, global plus scoped
+overrides); the default is `['timing', 'errors']`. Under the default, errors
+and timing rows are written; `warnings`-level and `all`-level verbosity require
+opt-in settings. A request's `x-log-level` header can raise verbosity for that
+call.
+
+## Reading logs
+
+`logs` (`.framework/functions/logs.ts`) is system-admin oriented (system
+admins query with the service client across accounts; account members get
+their RLS-scoped view):
+
+- `action=account` / `action=target` / `action=person` — filtered lists.
+- `action=search` — `ILIKE` message search.
+- `action=stats` — calls the `get_log_stats` RPC (totals, error count,
+  distinct levels/sources). **Note:** since migration `069`, when no
+  `p_date_from` is passed the window defaults to the **last 7 days** — a call
+  that omits the date range returns 7-day stats, not all-time stats.
+- `action=cleanup` — deletes rows older than `days_to_keep` (default 30).
+  `system-cron` also runs `cleanup_old_logs` on the 00:00 UTC tick, so
+  retention is enforced continuously.
+
+`observability` (`.framework/functions/observability.ts`) aggregates the same
+rows into dashboard metrics — actions `event_volume`, `error_rate`,
+`latency_percentiles`, `pipeline_stats`, `top_actors`, `ai_support_metrics`,
+`cleanup` — scoped to the caller's account unless the principal is
+system-admin.
+
+## Executions and deliveries
+
+Two sibling tables answer "did it run / did it arrive":
+
+- `public.pipeline_executions` — one row per pipeline run with `status`
+  (`queued` → `running` → `completed`/`failed`/`cancelled`), per-stage results,
+  `error_message`, `duration_ms`, and the `trigger_data` that caused it
+  (`.framework/functions/pipeline-executions.ts`; `action=running`,
+  `action=stats`, `action=cleanup`). Rows that never reach a terminal state
+  are swept to `failed` after 10 minutes.
+- `public.webhook_deliveries` — one row per inbound/outbound webhook attempt
+  with `direction`, `status` (`pending`/`processing`/`delivered`/`failed`),
+  `attempt_count`, `next_attempt_at`, `last_error`, `response_status`, and
+  `response_body` (`.framework/functions/deliveries.ts`; `action=retry`).
+
+`item-progress` (`.framework/functions/item-progress.ts`) is a different lens:
+per-person, per-item progress rows (`status`, `score`, `attempts`) that follow
+the type's `design_schema.pipeline` stage list (default
+`not_started → in_progress → completed`). Transitions must move forward
+unless `force: true` is passed; `started_at`/`completed_at` are stamped on
+first entry. Use it when you need per-user completion state rather than
+execution forensics.
+
+## Debugging a failed automation
+
+Work backwards through the tables:
+
+1. **`pipeline_executions`** — find the run. `status: 'failed'` +
+   `error_message` and per-stage results name the failing stage.
+   `status: 'queued'` past 10 minutes = dispatch never arrived (sweeper will
+   mark it failed); `status: 'running'` stuck = the runner died (same
+   sweeper).
+2. **`logs`** — search by `trace_id` (from the request) or `span_name`
+   (`pipeline.run`, `pipeline.stage.<handler>`, `db.<operation>`); audit rows
+   `trigger.fired` / `trigger.failed` / `pipeline.completed` /
+   `pipeline.failed` bracket the run. Check log levels if expected `timing`
+   rows are absent.
+3. **`webhook_deliveries`** — for outbound stages/integrations, the delivery
+   row carries `response_status`, `response_body`, `last_error`, and
+   `next_attempt_at`; retry with `action=retry`.
+4. **`api_key_usage_logs`** (`api-keys?action=usage-logs`) — whether the
+   machine caller ever arrived, and from where.
+5. **`observability` actions** — confirm the blast radius (`error_rate`,
+   `pipeline_stats`, `top_actors`) rather than chasing a single event.

package/.framework/src/instructions/concepts/permissions.md

@@ -0,0 +1,150 @@
+---
+title: Permission Model
+audience: developers, agents
+kind: concept
+verified_against: f69c37d
+verified_on: 2026-10-29
+source_files:
+  - .framework/functions/_shared/permissions.ts
+  - .framework/functions/_shared/principal.ts
+  - .framework/functions/_shared/db.ts
+  - .framework/functions/_shared/middleware.ts
+  - .framework/functions/admin-data.ts
+  - .framework/migrations/001_core_schema.sql
+  - .framework/migrations/002_core_rls_and_functions.sql
+  - .framework/migrations/003_core_system_data.sql
+---
+
+# Permission Model
+
+Spine authorization is two independent layers, and the code says so verbatim:
+**"RLS controls row access; design_schema controls what the principal can do.
+No permissions granted = none given."** (`.framework/functions/_shared/permissions.ts`)
+
+- **Row visibility (RLS):** Postgres policies scope every tenant table to the
+  accounts the caller can reach — see [Scope and Accounts](scope-and-accounts.md).
+- **Action authorization (`record_permissions`):** per request, the permission
+  engine evaluates the caller's role against the schema **stamped on the
+  record** to decide create/read/update/delete and per-field read/write.
+
+## Principals
+
+`.framework/functions/_shared/principal.ts` resolves two principal types:
+
+- **Human** — Supabase JWT → `people` row → `role_id` join to `roles`; the
+  principal carries `roles: [slug]` and gets a JWT-scoped DB client (`ctx.db`),
+  so RLS applies.
+- **Machine** — `x-api-key` header → `api_keys` row → `scopes` array and the
+  key's `account_id`. Machine principals get `adminDb` (service role), which
+  **bypasses RLS entirely**; their only authorization is the scope check.
+
+## The three surfaces
+
+`detectSurface` in `.framework/functions/_shared/permissions.ts` buckets the
+target by table name:
+
+| Surface | Covers | Rule |
+|---|---|---|
+| First (default) | runtime records — items, threads, messages, people, accounts, … | `record_permissions` on the record's stamped `design_schema` |
+| Second | config objects — `apps`, `pipelines`, `triggers`, `ai_agents`, `embeddings`, `timers`, `integrations`, `roles`, `types`, `prompt_configs` | `system_admin` full access; machine principals read-only; everyone else denied |
+| Third | system metadata — `logs`, `pipeline_executions`, `trigger_executions`, `link_types`, `links` | `system_admin`/machine full; humans read only their own/account rows |
+
+## First-surface evaluation
+
+`resolveFirstSurfacePermissions` runs on every runtime-record access:
+
+1. **Schema source is the record.** The stamped `design_schema` on the record
+   is authoritative; there is deliberately no fallback to the type row — a
+   record with no schema or no `record_permissions` is denied.
+2. **Role resolution.** The caller's role slug comes from
+   `people.role_id → roles.slug` (one role per person, cached ~60s). No role,
+   no permissions.
+3. **Record grant.** `record_permissions[roleSlug]` is an array of actions
+   (`create`, `read`, `update`, `delete`). The special key `'all'` is a
+   wildcard evaluated for every authenticated principal — grants to `all`
+   apply to anyone who passed RLS.
+4. **Field merge.** For each declared field, `fields.<name>.permissions[role]`
+   — an array drawn from `read`/`write` only — replaces the default. Fields
+   **without** an entry default to `{ read: canRead, write: canUpdate }` from
+   the record grant. That is why `record_permissions` is the base, most
+   restrictive grant and field entries are *overrides*: an explicit `["read"]`
+   on a field removes write access that `update` would otherwise give.
+
+> **Role keys are matched verbatim.** `record_permissions` keys must be real
+> `roles.slug` values — nothing normalizes them. The seeded admin slug is
+> `system_admin` (underscore, `.framework/migrations/003_core_system_data.sql`);
+> a `system-admin` key silently matches nobody and denies the admin.
+
+## Enforcement points
+
+- **Read** (`sanitizeFirstSurfaceRecordData`): `canRead` gates the record;
+  `data` is filtered per field by `fieldPermissions[name].read`; values are
+  formatted through `validation_schema` (`formatFieldData`); legacy `metadata`
+  is stripped from responses. Records with an empty `fields` map pass through
+  whole at the record grant.
+- **Update** (`validateFirstSurfaceUpdatePermissions`): `canUpdate` gates the
+  record, then **every key inside `data` must have `write`** — one denied field
+  fails the whole update with a per-field error. Values are sanitized through
+  `sanitizeFieldData` (type coercion + constraint checks; violations are
+  errors, not silent drops). Top-level non-`data` keys pass through the gate.
+  Writes therefore require both the record action *and* field-level write —
+  undeclared fields have no `fieldPermissions` entry and 403.
+- **Create** (`admin-data` `create`): permission is checked against the type's
+  `design_schema` (which becomes the record's stamp) before insert;
+  `type_id`/`type_slug` is required. A machine principal with `*:*` scope
+  bypasses the create check.
+
+## `system_admin` bypass
+
+`isSystemAdmin` (`roles` contains `system_admin`) short-circuits record
+access, sanitization, and update validation in `permissions.ts`, and gates the
+`types.ts`/`roles.ts`/`apps.ts` write APIs entirely. The same slug drives the
+RLS bypass inside `get_accessible_accounts` — system admins see all accounts.
+
+## Machine (API key) scopes
+
+`checkMachineScope` grants an action when the key's `scopes` contain
+`resource:action` exactly, `resource:*`, or `*:*` (e.g. `items:read`,
+`items:*`, `*:*`). Scopes are the *only* thing standing between a leaked API
+key and `adminDb`, so issue the narrowest scope that works — `*:*` also
+bypasses the create permission check in `admin-data`.
+
+## Canonical example
+
+The seeded pattern from the framework migrations — base grant first, field
+override only where needed:
+
+```json
+{
+  "record_permissions": {
+    "system_admin": ["create", "read", "update", "delete"],
+    "member_admin": ["create", "read", "update"],
+    "member":       ["create", "read", "update"]
+  },
+  "fields": {
+    "title":    { "data_type": "text",   "label": "Title",    "required": true,  "system": true },
+    "status":   { "data_type": "select", "label": "Status",   "required": true,
+                  "options": ["open", "closed"],
+                  "permissions": { "member": ["read"] } },
+    "internal_notes": { "data_type": "textarea", "label": "Internal Notes",
+                  "required": false,
+                  "permissions": { "member": [] } }
+  }
+}
+```
+
+Reading it: `member` can create and update records, so every declared field
+defaults to read+write — except `status`, whose `["read"]` override strips
+`member`'s write, and `internal_notes`, whose empty override hides it from
+`member` entirely. `member_admin` touches neither override, so it inherits
+record-level read+write on both. That is the whole discipline: **add a
+`fields.*.permissions` entry only where the base grant is wrong for that
+field** — usually to restrict a sensitive or `system: true` field, occasionally
+to loosen one the base grant closes.
+
+## See also
+
+- [Design Schema](design-schema.md) — where `record_permissions` lives
+- [Scope and Accounts](scope-and-accounts.md) — the RLS layer beneath this one
+- [../roles.md](../roles.md) — the role entity (`permissions` on roles bound
+  the *subject* side at the app/UI level)

package/.framework/src/instructions/concepts/runtime-records.md

@@ -0,0 +1,111 @@
+---
+title: Runtime Records
+audience: developers, agents
+kind: concept
+verified_against: f69c37d
+verified_on: 2026-10-29
+source_files:
+  - .framework/functions/admin-data.ts
+  - .framework/functions/_shared/resolve-ids.ts
+  - .framework/functions/_shared/permissions.ts
+  - .framework/migrations/001_core_schema.sql
+  - .framework/src/hooks/useEntityRecord.ts
+  - .framework/src/components/app-shell/GenericDetailPage.tsx
+---
+
+# Runtime Records
+
+Runtime records are the rows your users and apps actually create — as opposed
+to config entities (`types`, `apps`, `roles`, `pipelines`, …) that describe
+the system itself. They share one storage contract: a type, an account, a
+stamped schema, and a split between framework-owned columns and app-owned
+`data`.
+
+## The record kinds
+
+`admin-data` maps entity names to type `kind`s via `ENTITY_TO_KIND`
+(`.framework/functions/admin-data.ts`):
+
+| Entity (table) | `kind` | What it is |
+|---|---|---|
+| `items` | `item` | The workhorse — tickets, tasks, articles, any typed business object |
+| `threads` | `thread` | A conversation bound to a target (`target_type` + `target_id`); unique per `(target_type, target_id, visibility)` |
+| `messages` | `message` | An entry in a thread (`thread_id` + `sequence`, unique together) |
+| `links` | `link` | A polymorphic edge between two records (`source_type`/`source_id` → `target_type`/`target_id`, optional `link_type`) |
+| `attachments` | `attachment` | A stored file reference (filename, size, mime, `storage_path`) |
+| `watchers` | `watcher` | A person subscribed to a target (unique per target+person) |
+| `people` | `person` | Users/contacts — also carry `auth_uid`, `role_id`, `email` |
+| `accounts` | `account` | Tenants — the only table without `account_id` (it *is* the account) |
+| `item_progress` | `item_progress` | Progress/status snapshots for items |
+
+`VALID_ENTITIES` also accepts `logs`, `test_runs`, `observability`, and
+`test_cases` — the `RAW_ENTITIES` path: whitelisted raw-column inserts with no
+type requirement and no `design_schema`. `types` and `link_types` are accepted
+too but are config, not runtime records.
+
+## The shared row shape
+
+Every runtime table in `.framework/migrations/001_core_schema.sql` carries the
+same spine:
+
+- `id`, `created_at`, `updated_at`, `created_by` (+ `updated_by` on most)
+- `type_id uuid NOT NULL` — every record must reference a type
+- `account_id` — the owning tenant (all except `accounts`)
+- `app_id` — optional attribution to the app that created it
+- `data jsonb` — app-defined field values
+- `design_schema jsonb`, `validation_schema jsonb` — **stamped at creation**
+  from the type; the runtime source of truth for permissions and validation
+  (see [Design Schema](design-schema.md))
+- `is_active` — soft-delete flag; `remove` flips it rather than deleting
+
+`admin-data` create requires `type_id` *or* `type_slug` — `resolveTypeRecord`
+in `.framework/functions/_shared/resolve-ids.ts` resolves `(kind, slug)` to
+the type row, and the record is stamped before permission evaluation and
+insert.
+
+## `system: true` columns vs `record.data`
+
+The field declaration's `system` flag decides where a value lives:
+
+- **`system: true`** → a top-level record column (`record.title`,
+  `record.status`, `record.email`). These are framework-owned or
+  schema-critical fields.
+- **`system: false` / omitted** → inside `record.data.<field>`.
+
+Consumers split on the flag — this is the authoritative convention:
+
+- `useEntityRecord.save` partitions form data into `systemData` (top-level
+  keys) and `customData` (packed into `data`) by `fieldDef.system`
+  (`.framework/src/hooks/useEntityRecord.ts`).
+- `GenericDetailPage` initializes each field from `record[field]` when
+  `fieldDef.system` and `record.data[field]` otherwise
+  (`.framework/src/components/app-shell/GenericDetailPage.tsx`).
+- The permission engine's write gate evaluates every key inside `data`
+  against `fieldPermissions[name].write` and sanitizes through
+  `validation_schema` (`.framework/functions/_shared/permissions.ts`).
+
+**Fact:** send system fields as top-level keys and non-system fields inside a
+`data` object. A non-system field sent top-level is stored but invisible to
+schema-driven readers; a system field sent inside `data` never reaches its
+column.
+
+## Reading records back
+
+Reads pass through `sanitizeRecordData`: fields the caller can't read are
+dropped from `data`, the legacy `metadata` key is stripped from responses, and
+remaining values are formatted through `validation_schema`
+(`formatFieldData` — e.g. currency, multiselect, dates). A record the caller
+can't read at all collapses to `{id, created_at, updated_at}` — so "record
+exists but looks empty" is a permission answer, not missing data.
+
+## Writing rules of thumb
+
+- `type_slug` + `entity` on create; `id` on update/delete.
+- Never write `design_schema`, `validation_schema`, `type_id`, `account_id`,
+  or `id` in the body — create strips them before insert (`account_id` in the
+  body is honored as a *targeting* hint, see
+  [Scope and Accounts](scope-and-accounts.md)).
+- Deletes are soft (`is_active = false`) for the standard entities; only a few
+  entities hard-delete.
+- Creates fire triggers (`fireCreateTriggers`) and per-type rate limits after
+  permission checks — a custom insert path loses both.