spine-framework 0.4.3 → 0.5.0-rc.4

package/.framework/src/instructions/concepts/ai-and-prompts.md

@@ -0,0 +1,150 @@
+---
+title: "AI agents, prompt configs, and embeddings"
+audience: developers, agents
+kind: concept
+verified_against: f69c37d
+verified_on: 2026-10-29
+source_files:
+  - .framework/functions/ai-agents.ts
+  - .framework/functions/prompt-configs.ts
+  - .framework/functions/embeddings.ts
+  - .framework/functions/ai-interactions.ts
+  - .framework/functions/_shared/agent-runner.ts
+  - .framework/functions/_shared/llm.ts
+  - .framework/functions/_shared/pipeline-runner.ts
+  - .framework/migrations/034_match_embeddings_knn.sql
+  - .framework/migrations/044_match_embeddings_filtered_knn.sql
+---
+
+# AI agents, prompt configs, and embeddings
+
+Three config entities shape Spine's AI behavior:
+
+- An **AI agent** (`public.ai_agents`) is the *who*: a named model + system
+  prompt + tool set + guardrails.
+- A **prompt config** (`public.prompt_configs`) is the *how*: the prompt
+  template, model parameters, retrieval sources, and escalation policy applied
+  to a conversation.
+- An **embedding** (`public.embeddings`) is the *memory*: per-document vector
+  rows that semantic retrieval searches over.
+
+All three are ordinary admin-data-managed rows: `ai-agents` and
+`prompt-configs` expose CRUD via their own functions, and the conversational
+runtime lives in `.framework/functions/_shared/agent-runner.ts`.
+
+## Agent configuration
+
+`ai-agents` (`.framework/functions/ai-agents.ts`) stores, per row:
+
+- `name` + `agent_type` — required (`chat`, `analysis`, `automation`,
+  `custom` in the admin UI).
+- `model_config` — `{ model, temperature, max_tokens, tools }`. At inference
+  time the model resolves as `model_config.model || LLM_DEFAULT_MODEL ||
+  'gpt-4o'`; `model_config.tools` is passed through as OpenAI tool definitions
+  with `tool_choice: 'auto'`.
+- `system_prompt` — concatenated with the prompt config's `system_prompt`
+  (agent first) to form the system message.
+- `tools`, `capabilities`, `constraints`, `metadata` — `metadata
+  .default_prompt_config_id` is the fallback used when the thread doesn't
+  stamp its own `prompt_config_id`.
+- `is_active` — inactive agents fail `resolveAgentConfig` lookup.
+
+## Prompt configs and templating
+
+`prompt-configs` (`.framework/functions/prompt-configs.ts`) carries the fields
+the runner consumes per turn: `system_prompt`, `context_template`, `model`
+(default `gpt-4o`), `temperature` (0.7), `max_tokens` (4000),
+`is_multi_turn` (true), `max_history_messages` (20), `confidence_threshold`,
+`escalation_action` / `escalation_target`, `output_mode` / `output_field`,
+`requires_review`, `knowledge_sources`, `available_tools`, and
+`tool_constraints`.
+
+`context_template` supports exactly two placeholders, substituted in
+`buildContext`: `{{user_message}}` (the inbound text, truncated to 4000 chars)
+and `{{conversation_history}}` (prior messages rendered as
+`Customer:`/`Agent:` lines, truncated to 4000 chars).
+
+## How a thread resolves its agent and prompt
+
+`resolveAgentConfig` in `.framework/functions/_shared/agent-runner.ts`:
+
+1. Reads the thread (`threads`, joined with its `types` row).
+2. Agent id comes from `thread.data.agent_id`, falling back to the thread
+   type's `design_schema.default_agent_id`. No agent → the run aborts.
+3. Prompt config id comes from `thread.data.prompt_config_id`, falling back to
+   `agent.metadata.default_prompt_config_id`.
+4. Both rows must be `is_active`.
+
+`buildContext` then assembles the model input: combined system prompt, the
+rendered `context_template`, retrieved knowledge excerpts (numbered `[n]` and
+explicitly marked *untrusted* — the prompt instructs the model to treat them as
+evidence, never instructions), `## Conversation History`, `## Tool Results`,
+the filtered `available_tools` list, and finally `## Current Message` with a
+prompt-injection warning.
+
+## Invoking an agent
+
+The interactive path is `POST /api/ai-agents?action=run` with `{ thread_id,
+user_message, mode }` — `mode` is `step`, `complete`, or `episode` (default
+`complete`). Before any tokens are spent the handler runs a prompt-injection
+preflight (`detectInjection`); the app's `security.prompt_injection` setting
+chooses `off` | `flag` (default) | `block`. Each claimed run debits the
+`agent_step` rate limit.
+
+Inside a run, `runAgent`/`runAgentStep` build context, call inference, dispatch
+any `tool_calls` the model returns through `executeToolAction`, persist the
+outbound `messages` row stamped with `agent_id` and `prompt_config_id`, and
+manage the episode lease (see [concurrency](./concurrency-and-idempotency.md)).
+
+Pipelines reach AI through stage types, not the `run` action:
+
+- `agent_inference` — POSTs `{ context, model, temperature, max_tokens }` to
+  `config.webhook_url` via `integration-outbound` (`generic-http`); without a
+  `webhook_url` it returns a mock response.
+- `search_knowledge` — retrieval over knowledge sources for the stage config's
+  `query`.
+- `generate_embeddings` / `revoke_embeddings` / `kb_ingest` / `kb_revoke` /
+  `create_interaction` — delegate to the `embeddings`, `kb`, and
+  `ai-interactions` functions described below.
+
+## The LLM call itself
+
+`_shared/llm.ts` does not hold API keys. `callOpenAI` routes every request
+through `integration-outbound` with `integration_slug: 'openai'` and
+`credential_field: 'api_token'` — credentials come from the `integrations`
+table like any other outbound call. `callOpenAIChat` defaults to
+`gpt-4o-mini`/0.7/4000 tokens, overridable per call and via
+`integration.config.models.chat` / `.embedding`. When no OpenAI integration or
+credential exists, inference returns a clearly-marked mock response rather than
+failing.
+
+## Embeddings and retrieval
+
+`embeddings` (`.framework/functions/embeddings.ts`) manages the
+`public.embeddings` vector store:
+
+- `action=generate` builds embeddable text per **vector type**. The type's
+  `design_schema.vector_types` decides which builders run — default
+  `['semantic', 'structure']`; registered builders also cover `intent`,
+  `cause`, `procedure`, `outcome`, `context`, and `error_signal`. Each vector
+  type occupies a fixed `chunk_index` slot so concurrent types never collide
+  on the unique constraint, and unchanged content is skipped by hash.
+- `action=revoke` sets `revoked_at` on a document's vectors; `action=approve`
+  sets `approved_at` and clears revocation.
+- `action=search-similar` calls the `match_embeddings` RPC (pgvector KNN,
+  defined in migrations `034`/`044`) with `query_embedding`, `match_count`,
+  `similarity_threshold`, and optional account/vector-type/item-type/document
+  filters.
+- `action=search-semantic` is a Postgres `websearch` full-text query over the
+  `content` column — the lexical complement to vector search.
+
+## Interaction records
+
+`ai-interactions` (`.framework/functions/ai-interactions.ts`) writes the
+feedback/observability side of the loop as `items` rows (default type
+`ai_feedback_record`, overridable via `type_slug`; the call returns
+`{ skipped: true }` when the type isn't installed). Actions:
+`submit_feedback`, `create_interaction` (what the pipeline stage calls — it
+stamps `thread_id`, `ai_message_id`, `retrieved_sources`, `ai_confidence`,
+`kb_candidate`, and review fields), `update_interaction`,
+`create_cil_proposal`, and `list`.

package/.framework/src/instructions/concepts/apps-and-renderers.md

@@ -0,0 +1,162 @@
+---
+title: Apps and Renderers
+audience: developers, agents
+kind: concept
+verified_against: f69c37d
+verified_on: 2026-10-29
+source_files:
+  - .framework/migrations/001_core_schema.sql
+  - .framework/migrations/013_get_account_apps_role_filter.sql
+  - .framework/migrations/014_apps_required_roles.sql
+  - .framework/src/App.tsx
+  - .framework/src/components/AppWrapper.tsx
+  - .framework/src/components/CustomAppLoader.tsx
+  - .framework/src/components/app-shell/GenericAppShell.tsx
+  - .framework/src/components/app-shell/GenericListPage.tsx
+  - .framework/src/components/app-shell/GenericDetailPage.tsx
+  - .framework/functions/_shared/app-manifest.ts
+  - .framework/functions/apps.ts
+  - .framework/cli/commands/create-app.ts
+  - .framework/cli/commands/install-app.ts
+---
+
+# Apps and Renderers
+
+An app in Spine is a **row in `public.apps`** plus — for apps that render their
+own UI — a directory of code. The registry row decides where the app mounts,
+who can open it, and *how* it renders. The `renderer` column is the fork in
+the road.
+
+## The registry row
+
+`.framework/migrations/001_core_schema.sql` defines `public.apps`: unique
+`slug`, `name`, `route_prefix`, `renderer` (CHECK constraint:
+`'generic' | 'custom' | 'none'`), `min_role`, `required_roles` (added in
+migration 014), `nav_items`, `config`, `integration_deps`, `metadata`,
+`is_active`, `is_system`, `owner_account_id`, domains.
+
+- `route_prefix` (defaults to `/<slug>`) is where the SPA mounts the app —
+  `.framework/src/App.tsx` builds a `<Route>` per active app.
+- `is_active: false` removes the app from navigation and routing.
+- `is_system: true` protects the row from tenant-level deletion.
+
+## Role gating
+
+Two layers decide who sees an app:
+
+- Server side, `get_account_apps` filters the app list by the caller's role
+  slugs (`.framework/functions/apps.ts`;
+  `.framework/migrations/013_get_account_apps_role_filter.sql`).
+- Client side, `AppWrapper` computes `required_roles` (falling back to
+  `[min_role]`) and renders an access-denied screen unless the user holds
+  `system_admin` or one of those roles (`.framework/src/components/AppWrapper.tsx`).
+
+This gates *opening the app*; it does not replace record-level
+`record_permissions` on the data underneath.
+
+## `renderer = 'generic'`
+
+The platform renders the app's list and detail pages for you, straight from
+each type's `design_schema.views`:
+
+- `App.tsx` mounts `GenericAppShell` for `renderer === 'generic'` apps.
+- `GenericAppShell` builds the sidebar from the app's `nav_items` and, for
+  every nav item carrying `type_slug` (and optional `view`), generates
+  list/detail/new routes backed by `GenericListPage` and
+  `GenericDetailPage`.
+- `GenericListPage` turns `views.default_list` (or the nav item's named view)
+  into sortable columns; `GenericDetailPage` renders `views.default_detail`
+  sections with per-field read/write awareness.
+
+**What you get for free:** a working list page, detail page, create/edit form,
+sorting, filters, field-type rendering, and permission-aware editing — with
+zero app code. **What it costs:** every type the app serves must have complete
+`fields`, `views.default_list`, and `views.default_detail`, or the pages fail
+with "No schema available" / schema-load errors. See
+[Design Schema](design-schema.md).
+
+## `renderer = 'custom'`
+
+The app ships its own React code under `custom/apps/<slug>/` (a
+project-owned directory, not part of the framework package). `App.tsx` mounts
+`CustomAppLoader`, which resolves the app through an
+`import.meta.glob('custom/apps/*/index.tsx')` — the app's `index.tsx` is its
+entry point and default-exports the routed component (the `create-app`
+scaffold generates exactly this shape).
+
+A custom app renders its own pages but still uses the platform for data:
+`/api/admin-data` for runtime CRUD, `/api/types` for schemas. Custom apps
+must never write their own CRUD wrappers — that bypasses permissions,
+validation, audit, and triggers (see [Runtime Records](runtime-records.md)).
+
+`renderer = 'none'` registers the app (data, types, integrations) with no UI
+surface.
+
+## The manifest
+
+`custom/apps/<slug>/manifest.json` is the app's declarative half, validated by
+`loadManifest` in `.framework/functions/_shared/app-manifest.ts` (the
+`AppManifest` interface is the contract):
+
+```json
+{
+  "name": "My App",
+  "slug": "my-app",
+  "description": "…",
+  "version": "1.0.0",
+  "required_roles": ["member"],
+  "route_prefix": "/my-app",
+  "renderer": "custom",
+  "routes": ["/my-app", "/my-app/dashboard"],
+  "nav_items": [{ "title": "Dashboard", "path": "/my-app/dashboard", "icon": "LayoutDashboard", "order": 1 }],
+  "features": [],
+  "dependencies": [],
+  "entry_point": "./index.tsx",
+  "is_public": false,
+  "auth_required": true
+}
+```
+
+Validated requirements: non-empty `name`, `slug` matching
+`^[a-z][a-z0-9-]*$` (and equal to the directory name for discovery), and
+`required_roles` as an array of strings. Optional keys the install flow also
+honors: `config` (merged onto the apps row), `seed` (declared seed entries),
+`owned_types`, `registration`, `metadata`, `min_role`,
+`preview_domain`/`prod_domain`.
+
+## Install and seed flow
+
+Two paths put an app on disk and in the registry:
+
+- **`npx spine-framework create-app <slug>`** — scaffolds
+  `custom/apps/<slug>/` (`manifest.json` + `index.tsx` + `components/`) and
+  inserts a `public.apps` row with `renderer: 'custom'` and
+  `route_prefix: /<slug>` (`.framework/cli/commands/create-app.ts`).
+- **`npx spine-framework install-app <package>`** — copies a packaged app into
+  `custom/apps/<slug>/`, runs migrations, then `seedApp`:
+  - upserts the `apps` row from the manifest (`appRowPayload`: name,
+    description, version, nav_items, route_prefix, renderer, required_roles,
+    min_role, config merge, metadata);
+  - applies seed files in dependency order — conventional
+    `seed/roles.json` → `roles` (conflict `slug`), `seed/types.json` →
+    `types` (conflict `kind,slug`, **no** `app_id` injection,
+    `ignoreDuplicates`) — plus any `manifest.seed` entries;
+  - refuses to overwrite a `types` row whose `design_schema` differs from the
+    seed unless the slug is declared in `manifest.owned_types` and
+    `--reconcile-owned-types` was passed (types are global — see
+    [Scope and Accounts](scope-and-accounts.md));
+  - syncs the `registration` block into `spine.config.json` (the project-owned
+    config file at the project root).
+- **`update-db-app <slug>`** re-syncs a local `manifest.json` to the DB without
+  touching files; **`seed-app <slug>`** re-runs seeds;
+  **`uninstall-app`** removes the app (`--keep-data` preserves seeded rows).
+  All in `.framework/cli/commands/install-app.ts`.
+
+## The practical split
+
+Use `generic` when the app is "records in, records out" — lists, detail forms,
+admin-style editing — and keep views accurate. Use `custom` when the UX is a
+product surface (dashboards, wizards, bespoke layouts) and treat the manifest
+as your app's contract: registration, roles, seeds, nav. Either way, types and
+`design_schema` do the data modeling — the renderer only decides who draws the
+pixels.

package/.framework/src/instructions/concepts/auth-and-api-keys.md

@@ -0,0 +1,133 @@
+---
+title: "Auth: sessions, API keys, and invites"
+audience: developers, agents
+kind: concept
+verified_against: f69c37d
+verified_on: 2026-10-29
+source_files:
+  - .framework/functions/_shared/principal.ts
+  - .framework/functions/_shared/middleware.ts
+  - .framework/functions/_shared/hash-api-key.ts
+  - .framework/functions/_shared/rate-limit.ts
+  - .framework/functions/api-keys.ts
+  - .framework/functions/invites.ts
+  - .framework/functions/integration-routes.ts
+  - .framework/migrations/035_rate_limits.sql
+  - .framework/migrations/052_registration_security.sql
+---
+
+# Auth: sessions, API keys, and invites
+
+Every request to a Spine function resolves to one **Principal**
+(`_shared/principal.ts`): either `type: 'human'` (a `people` row + roles) or
+`type: 'machine'` (an `api_keys` row + scopes), plus `ANONYMOUS_PRINCIPAL` and
+`SYSTEM_PRINCIPAL` sentinels. `resolvePrincipal` is the single dispatch point,
+run by `createHandler` / `createPublicHandler` in `_shared/middleware.ts`:
+
+1. `x-api-key` header → **machine** principal.
+2. A valid `INTERNAL_INVOKE_SECRET` (function-to-function) → `x-cron-id` /
+   `x-trigger-id` impersonation, else `SYSTEM_PRINCIPAL`. These headers are
+   honored only alongside the secret — a bare id is ignored.
+3. `Authorization: Bearer <jwt>` → **human** principal via Supabase auth.
+4. Nothing → `ANONYMOUS_PRINCIPAL` (machine, no scopes, no account).
+
+The principal also selects the database client (`getPrincipalDb`): humans get a
+JWT-scoped client (row-level security applies); machines and anonymous get the
+service-role client, so endpoint code is responsible for checking machine
+scopes — `machineHasScope` honors `resource:action`, `resource:*`, and `*:*`,
+and `admin-data` treats a machine with `*:*` as authorized (with role
+`system_admin`) while lesser keys still pass through record permissions.
+
+## Browser sessions (humans)
+
+`Bearer <supabase-jwt>` → `supabase.auth.getUser` → `resolveInternalPersonId`
+binds the auth user to a `people` row by `auth_uid` (email fallback never
+binds unclaimed invite stubs), then loads the person with its role join. The
+principal carries `roles: [role.slug]`, `accountId`, and the raw JWT in
+`authContext` (never serialized — `formatPrincipalForAudit` omits it). Results
+cache for 60 seconds per warm instance, so role/account changes take up to a
+minute to propagate; a person with neither `account_id` nor `role_id`
+(registration not yet placed) is never cached.
+
+A human's effective rights come from `roles` → `isSystemAdmin` /
+`humanHasRole` → the `design_schema.record_permissions` on each type. An
+invalid or expired JWT silently becomes anonymous — endpoints then fail with
+"Authentication required" / "Account context required" rather than 401s.
+
+## API keys (machines)
+
+Mint a key with `POST /api/api-keys` (`.framework/functions/api-keys.ts`):
+`{ name, key_type?, scopes?, rate_limit?, expires_at?, integration_id?,
+app_id?, machine_type?, is_internal? }`. The response returns `raw_key` —
+`sk_<32 hex>` — **once**, with a "store this now" warning. Only the first 8
+characters (`key_prefix`, the lookup index) and a PBKDF2-SHA256 hash
+(100k iterations, `pbkdf2$iterations$salt$hash`) are stored; `omitSecretFields`
+strips secret material from every read response.
+
+Use it as `x-api-key: sk_...` on any function endpoint. Resolution:
+
+- Prefix-matched candidate keys must be `is_active` and unexpired, then the
+  hash is verified with a timing-safe compare.
+- Per-key rate limit: `rate_limit` (default 100) requests per
+  `config.rate_limit_window_seconds` (default 60), counted in
+  `api_key_usage_logs`.
+- Per-IP ceiling: `rate_limit_hit` RPC on `ip:<ip>:<path>`, default 300/minute
+  (`SPINE_IP_RATE_LIMIT`), fail-open.
+- Every request writes an `api_key_usage_logs` row (method, path, IP,
+  user-agent) — `action=usage-logs` on the same endpoint reads them back.
+
+The resulting machine principal inherits `key.scopes`, `key.machine_type`
+(default `'integration'`), `key.is_internal`, and `key.account_id`. A `*:*`
+scope acts as `system_admin`. `integration-routes` (inbound webhooks)
+*requires* a machine principal — a JWT cannot deliver webhooks — and keys
+pinned to `integration_id`/`app_id` are restricted to those integrations.
+Related actions: `action=validate` (verify a key + check permissions),
+`action=revoke` (sets `is_active=false`), `action=usage-logs`.
+
+Shared throttling also applies: `checkRateLimit(ctx, action)`
+(`_shared/rate-limit.ts`) resolves named policies from
+`accounts.data.rate_limits[action]` (account override) or the calling app's
+`config.rate_limits` (with `rate_limit_profile` indirection), and enforces
+them through the atomic `rate_limit_hit` RPC on `public.rate_limits`
+(migration `035`). Over-limit → 429 (`RateLimitError`); a `fail_closed`
+policy → 503 when the counter RPC is unavailable.
+
+## First admin and invites
+
+Bootstrap is a Postgres trigger, not an endpoint: on the first `auth.users`
+signup, `handle_new_auth_user` (migrations `010`, hardened in `052`) takes an
+advisory lock, creates the `people` row, and — only when **no** `people` rows
+exist — grants the `system_admin` role under the `spine-system` account.
+Serialization means two concurrent first signups cannot both become
+system-admin; `system_admin` can never be assigned through registration.
+
+Everyone after that arrives by invite (`.framework/functions/invites.ts`):
+
+- `action=create-invite` — requires `system_admin` or an `*_admin` role on the
+  target account; `role_slug: 'system_admin'` is refused. Creates a stub
+  `people` row holding `data.invite_token` (16-char) and
+  `invite_expires_at` (7 days), returns
+  `{APP_BASE_URL}/register?t=<token>`, and can send the email through a
+  configured email integration.
+- `action=validate-token` (public) — checks the token exists, is unblocked,
+  and unexpired.
+- `action=complete-registration` — after Supabase signup, calls the
+  service-role-only `complete_registration_placement` RPC, which atomically
+  stamps `account_id` + `role_id` from the app's server-side registration
+  config (`new_account`, `existing_account`, `join_request` paths). Client
+  claims about role or target are never trusted — the RPC is the only
+  placement writer.
+- Admin management: `list-invites`, `revoke-invite`, `reinvite`,
+  `block-invite`, `approve`, `deny`, `list-members`, `update-member`,
+  `deactivate-member`. `check-first-user` and `check-domain` are public
+  read-only probes used by the registration UI.
+
+## What each surface can reach
+
+| | Browser JWT | API key | Internal secret |
+|---|---|---|---|
+| Principal | human (roles) | machine (scopes) | system / cron / trigger |
+| DB client | JWT-scoped (RLS) | service role | service role |
+| `admin-data`, entity APIs | role + record permissions | scopes; `*:*` = system admin | full |
+| `integration-routes` (inbound) | no | required | — |
+| `system-cron` / `trigger-scheduler` | no | `SCHEDULER_API_KEY` or internal machine | yes |

package/.framework/src/instructions/concepts/automation.md

@@ -0,0 +1,186 @@
+---
+title: "Automation: pipelines, triggers, and timers"
+audience: developers, agents
+kind: concept
+verified_against: f69c37d
+verified_on: 2026-10-29
+source_files:
+  - .framework/functions/pipelines.ts
+  - .framework/functions/pipeline-executions.ts
+  - .framework/functions/pipeline-runner-background.ts
+  - .framework/functions/_shared/pipeline-runner.ts
+  - .framework/functions/_shared/trigger-engine.ts
+  - .framework/functions/timers.ts
+  - .framework/functions/triggers.ts
+  - .framework/functions/trigger-scheduler.ts
+  - .framework/functions/system-cron.ts
+  - .framework/functions/admin-data.ts
+  - .framework/migrations/001_core_schema.sql
+  - .framework/migrations/018_runtime_tables.sql
+  - .framework/migrations/020_schema_reconciliation.sql
+  - .framework/migrations/068_cron_schedule_timer_rpcs.sql
+  - netlify.toml
+---
+
+# Automation: pipelines, triggers, and timers
+
+Spine automation is three primitives plus a ledger:
+
+- A **pipeline** (`public.pipelines`) is an ordered JSON `stages` array — the
+  definition of *what runs*.
+- A **trigger** (`public.triggers`) maps a record event
+  (`<kind>_created` / `<kind>_updated` / `<kind>_deleted`) to a pipeline — *when
+  data changes*.
+- A **timer** (`public.timers`) points at a pipeline and carries
+  `next_execution` — *when the clock says so*.
+- A **pipeline execution** (`public.pipeline_executions`) is the durable record
+  of one run — *what actually happened*.
+
+There is no separate "workflow engine" service. Everything runs inside the
+Netlify functions, coordinated through Postgres rows.
+
+## Pipelines and the runner
+
+`runPipeline` in `.framework/functions/_shared/pipeline-runner.ts` is the single
+executor. It loads the pipeline row (inactive or missing pipelines throw),
+creates or claims a `pipeline_executions` row, then walks `pipeline.stages` in
+array order.
+
+Each stage names a `stage_type`. That value is **not** a function name — it is a
+slug looked up in `public.actions` (`eq('slug', stage.stage_type)`,
+`is_active = true`). The action row supplies `handler_module` and `handler`;
+the stage's `config` is merged over `action.config` along with internal keys
+`_pipelineId`, `_executionId`, `_stageIndex`, and `_triggerData`. The runner
+then routes on `handler_module`:
+
+- `functions` — dispatch to a named handler in the runner's `stageHandlers`
+  map. Built-ins include `update_item`, `create_record`, `http_request`,
+  `send_notification`, `send_email`, `run_pipeline`, `query_items`,
+  `search_knowledge`, `agent_inference`, `generate_embeddings`,
+  `revoke_embeddings`, `kb_ingest`, `kb_revoke`, and `create_interaction`.
+- `integrations` — dispatch through `integration-outbound` (the same path
+  outbound webhooks use).
+- `webhook` — POST/GET to `action.config.url` via `integration-outbound`.
+
+Stage config strings support `{{trigger.*}}` and `{{record.*}}` placeholders,
+resolved recursively against the trigger envelope and the firing record's data
+(`record` is `triggerData.entity.data`). Timer- and manual-fired runs have **no
+record context** — `{{record.*}}` placeholders pass through unresolved and
+appear literally in stage output (e.g. notification text). Author timer-driven
+stages against `{{trigger.*}}` or fixed text.
+
+A stage that throws is recorded as `failed` in the stage results; the run then
+aborts **unless** the stage sets `continue_on_error: true`. Completed and
+failed runs both write terminal status, stage results, and duration back to the
+execution row — `pipeline-executions` is therefore the ground truth for "did it
+work", and `status: 'cancelled'` is available for operator-initiated stops.
+
+## Triggers: record events
+
+Writes that go through `admin-data` call into
+`.framework/functions/_shared/trigger-engine.ts`: creates fire
+`fireCreateTriggers`, updates fire `fireUpdateTriggers` with the previous row
+snapshot. (The delete path currently does not invoke `fireDeleteTriggers` —
+`*_deleted` event types exist in the union, but nothing emits them from
+`admin-data`.) Trigger fires are caught-and-logged, so a failing trigger never
+rolls back the write that caused it.
+
+`checkAndFireTriggers` loads active triggers for the event type and evaluates
+each against `config`:
+
+- `config.entity_type` must match the runtime table's kind (`items` → `item`,
+  `people` → `person`, and so on).
+- `config.type_slug` restricts to one `types.slug`; the engine resolves the
+  slug from the record's `type_id` when needed.
+- `config.filters` is a map of path → expected value. Paths support dot
+  notation and Postgres-style `data->>'field'` accessors; expected values may
+  be literals, arrays (membership), or operator objects using `$eq`, `$ne`,
+  `$gt`, `$gte`, `$lt`, `$lte`, `$in`, `$nin`, `$exists`.
+- `config.fire_on_transition` makes an update trigger fire only when the record
+  *moves into* the matching state — the previous snapshot must not already
+  match. This is what prevents "every save to a published record re-fires the
+  pipeline".
+
+A matching trigger does **not** run the pipeline inline. It:
+
+1. Inserts a `pipeline_executions` row with `status: 'queued'` (a stable
+   execution id exists before any work starts).
+2. POSTs to `/.netlify/functions/pipeline-runner-background` with an
+   `X-Internal-Secret` header, awaiting only the dispatch ack (bounded by a 5s
+   timeout) — the caller's request never waits for the pipeline to finish.
+3. Records `record_trigger_fire` bookkeeping and emits a `trigger.fired` audit
+   event (or `trigger.failed` on error).
+
+The background function verifies `INTERNAL_INVOKE_SECRET`, reconstructs a
+machine context, and calls `runPipeline` with the pre-created `execution_id`,
+which claims the row `queued → running` instead of inserting a duplicate.
+
+## Timers and the scheduler
+
+A timer row carries `timer_type`, `pipeline_id`, `config`, and
+`next_execution`. The `get_due_timers` RPC returns active timers whose
+`pipeline_id` is set and `next_execution <= now()`; `update_timer_after_run`
+stamps `last_execution`, increments `execution_count`/`success_count`/
+`failure_count`, and reschedules `next_execution` from
+`config.interval_minutes` — a timer with no positive `interval_minutes` gets
+`next_execution = NULL` and never runs again.
+
+The clock is `trigger-scheduler`, a Netlify scheduled function configured for
+`* * * * *` (every minute) in `netlify.toml`. It accepts genuine Netlify
+Clockwork invocations (schedule header or `Netlify Clockwork` user-agent *plus*
+a `{"next_run": ...}` body) and fabricates a system-principal context;
+non-Clockwork calls fall through to the cron handler's own auth, which accepts
+either `?api_key=$SCHEDULER_API_KEY` or an internal machine principal — anything
+else gets 403 and an audit event.
+
+Each tick, `system-cron` does all of the following:
+
+- **Sweeps stale executions** — `sweepStaleExecutions` marks
+  `running`/`pending`/`queued` rows untouched for over 10 minutes as `failed`.
+  This is the safety net for a runner that died mid-flight or a dispatch that
+  never arrived.
+- **Runs due schedules** — `get_due_schedules` → validate the creator → load
+  the `public.actions` row → execute as the schedule's delegated machine
+  principal (`schedules`/`actions`/`schedule_executions` are the runtime
+  scheduling tables).
+- **Runs due timers** — each due timer runs its pipeline as machine principal
+  `timer:<timer_id>` with scope `pipelines:execute`, then
+  `update_timer_after_run` reschedules it.
+- **Evaluates threshold alerts** — `threshold_alerts` rows compare
+  `error_rate`, `latency_p95`, or `pipeline_failure_rate` metrics against a
+  value and fire the alert's `pipeline_id` on breach.
+- **Retries webhook deliveries** — `retryPendingDeliveries` runs every tick.
+- **Cleans logs** — `cleanup_old_logs` runs on the 00:00 UTC tick.
+
+## Observing executions
+
+- `pipeline-executions` (`.framework/functions/pipeline-executions.ts`):
+  `GET` list/get, `action=running`, `action=stats`, `action=cleanup`
+  (`days_to_keep` defaults to 30), plus `start`/`complete`/`cancel`.
+- `pipelines` (`.framework/functions/pipelines.ts`): CRUD plus
+  `action=by-trigger`, `action=executions`, `action=toggle`.
+- `triggers` and `timers` (`.framework/functions/triggers.ts`,
+  `.framework/functions/timers.ts`): CRUD plus `toggle`; triggers also expose
+  `listByEvent` and `getExecutions`.
+- CLI: `spine-framework pipelines run <id>` executes a pipeline;
+  `spine-framework pipelines executions <id>` lists recent runs
+  (`.framework/cli/commands/pipelines.ts`).
+
+An execution stuck in `queued` means dispatch failed before the background
+function claimed it — the sweeper fails it after 10 minutes. An execution stuck
+in `running` means the runner died — same sweeper, same outcome.
+
+## How the pieces connect
+
+1. A client writes a record through `admin-data`; `fireCreateTriggers` /
+   `fireUpdateTriggers` run in-process after the write.
+2. The trigger engine matches the event to active `triggers` rows, inserts a
+   `queued` execution, and dispatches to `pipeline-runner-background`.
+3. Independently, Netlify Clockwork invokes `trigger-scheduler` every minute;
+   `system-cron` runs due timers and schedules through the same `runPipeline`,
+   sweeps dead executions, retries deliveries, and evaluates alert thresholds.
+4. `runPipeline` resolves each `stage_type` through `public.actions` and runs
+   the stage handlers — which is also where outbound integrations, AI agent
+   calls, and embedding jobs enter the system.
+5. Every step lands in `pipeline_executions`, `logs`, and the audit trail,
+   which is what the admin UI and `pipeline-executions` API read.