create-restforge-skills 0.2.0 → 0.4.0

package/skills/restforge/references/rdf-advanced.md

@@ -1,488 +1,695 @@
-# Reference: RDF Advanced Features
-
-> **Offline mirror.** This file mirrors the RDF catalog of the installed
-> RESTForge platform (`restforge-handbook/catalogs/rdf/`). The live platform is
-> authoritative — when this file and the platform disagree, trust the platform,
-> then update this file. For `fieldValidation` constraints, see
-> `references/field-validation.md`.
-
-This reference covers advanced RDF payload features beyond standard CRUD fields.
-
----
-
-## Table of Contents
-
-1. [Data Source Resolution](#data-source-resolution)
-2. [Query File Reference](#query-file-reference)
-3. [Field Lookup](#field-lookup)
-4. [Default Scope](#default-scope)
-5. [Workflow (Change-Status)](#workflow-change-status)
-6. [Master-Detail (Composite)](#master-detail-composite)
-7. [Aggregate Config](#aggregate-config)
-8. [Adjust Config](#adjust-config)
-9. [Import Config](#import-config)
-10. [Processor](#processor)
-11. [Kafka Event Publishing](#kafka-event-publishing)
-12. [Components (Lifecycle Hooks)](#components-lifecycle-hooks)
-
----
-
-## Data Source Resolution
-
-RESTForge resolves data sources per endpoint using a priority chain. Define
-only what is needed; the platform falls back automatically.
-
-| Endpoint | Resolution order |
-|---|---|
-| `/datatables` | `datatablesQuery` → `SELECT * FROM tableName` |
-| `/read`, `/first`, `/lookup` | `viewName` → `viewQuery` → `tableName` |
-| `/export` | `exportQuery` → `SELECT {fields} FROM tableName` |
-| `/read-composite` (detail) | `detailQuery` → detail `tableName` |
-
-**`viewName`** — reference a database VIEW:
-```json
-"viewName": "v_order_summary"
-```
-
-**`viewQuery`** — inline SQL (virtual view, no DB object created):
-```json
-"viewQuery": "SELECT o.*, c.customer_name FROM orders o JOIN customers c ON o.customer_id = c.customer_id"
-```
-
-**`datatablesQuery`** — SQL for the paginated table with `:search`, `:sort`,
-`:limit`, `:offset` placeholders:
-```json
-"datatablesQuery": "SELECT o.*, c.customer_name FROM orders o JOIN customers c ON o.customer_id = c.customer_id WHERE 1=1"
-```
-
-Write source is always `tableName` — `viewName`/`viewQuery` are read-only.
-
----
-
-## Query File Reference
-
-SQL queries can be stored in external `.sql` files using the `file:` prefix.
-Path is relative to the payload file location.
-
-```json
-"datatablesQuery": "file:sql/orders-datatables.sql",
-"exportQuery": "file:sql/orders-export.sql"
-```
-
-Convention for folder structure:
-```
-payload/
-├── order.json
-└── sql/
-    ├── orders-datatables.sql
-    └── orders-export.sql
-```
-
-External SQL files support the same placeholders as inline queries.
-For master-detail, each detail query is in a separate file.
-
----
-
-## Field Lookup
-
-Configures dropdown/autocomplete data for a field. Used for foreign key fields
-that need a human-readable label.
-
-```json
-{
-  "fieldName": "category_id",
-  "type": "string",
-  "fieldLookup": {
-    "apiPath": "/category",
-    "id": "category_id",
-    "text": "category_name"
-  }
-}
-```
-
-- `apiPath` — backend resource that provides lookup options via `/lookup` endpoint.
-- `id` — field returned as the stored value.
-- `text` — field returned as the display label.
-
-Static lookup (no API call):
-```json
-"fieldLookup": {
-  "type": "static",
-  "options": [
-    { "id": "A", "text": "Option A" },
-    { "id": "B", "text": "Option B" }
-  ]
-}
-```
-
----
-
-## Default Scope
-
-Automatic WHERE clause injected on `/lookup` and `/read`-family endpoints.
-Used for tenant isolation, user-scoped data, or active record filtering.
-
-```json
-"defaultScope": {
-  "actions": ["lookup", "read", "datatables"],
-  "conditions": [
-    { "key": "is_active", "value": true },
-    { "key": "company_id", "value": ":companyId" }
-  ]
-}
-```
-
-- `actions[]` — which endpoints apply the scope.
-- Conditions with `:paramName` resolve from the request context (e.g., JWT claims).
-- Combines with user-supplied WHERE via AND.
-- The `is_active` column is auto-synced by the processor's `.active()` method.
-
----
-
-## Workflow (Change-Status)
-
-Adds a `/change-status` endpoint with state machine validation.
-
-```json
-"workflow": {
-  "statusField": "status",
-  "transitions": [
-    {
-      "from": "draft",
-      "to": "submitted",
-      "action": "submit",
-      "onBefore": "http://internal-service/validate",
-      "onAfter": "http://notification-service/notify"
-    },
-    {
-      "from": "submitted",
-      "to": "approved",
-      "action": "approve"
-    },
-    {
-      "from": ["submitted", "approved"],
-      "to": "rejected",
-      "action": "reject"
-    }
-  ]
-}
-```
-
-| Property | Notes |
-|---|---|
-| `statusField` | Field name that holds the current status |
-| `transitions[].from` | Current status (string or array of strings) |
-| `transitions[].to` | Target status after transition |
-| `transitions[].action` | Action identifier in the request body |
-| `transitions[].onBefore` | HTTP call before transition; 4xx/5xx blocks the transition (HTTP 422/502) |
-| `transitions[].onAfter` | HTTP call after transition; failure does not roll back |
-
----
-
-## Master-Detail (Composite)
-
-Adds `/create-composite`, `/update-composite`, and `/read-composite` endpoints.
-
-```json
-"details": [
-  {
-    "tableName": "order_item",
-    "foreignKey": "order_id",
-    "primaryKey": "item_id",
-    "fields": [
-      { "fieldName": "product_id", "type": "string" },
-      { "fieldName": "qty", "type": "integer" },
-      { "fieldName": "price", "type": "decimal" }
-    ]
-  }
-]
-```
-
-- Multiple entries in `details[]` generate multiple detail tabs.
-- `foreignKey` links detail rows to the master record.
-- Detail `fields[]` follow the same validation rules as master fields.
-- `/update-composite` supports three detail operations in one call:
-  `insert` (new rows), `update` (changed rows), `delete` (removed rows).
-- `/read-composite` returns the master record with all detail arrays nested.
-
----
-
-## Aggregate Config
-
-Adds an `/aggregate` endpoint for COUNT, SUM, AVG, MIN, MAX operations.
-
-```json
-"aggregateConfig": {
-  "joins": [
-    {
-      "type": "LEFT",
-      "table": "category",
-      "on": "product.category_id = category.category_id"
-    }
-  ],
-  "groupBy": ["category_name"],
-  "operations": [
-    { "function": "COUNT", "field": "product_id", "alias": "total_products" },
-    { "function": "SUM", "field": "stock_qty", "alias": "total_stock" },
-    { "function": "AVG", "field": "price", "alias": "avg_price" }
-  ]
-}
-```
-
-| Operation | Description |
-|---|---|
-| `COUNT` | Count rows or non-null values |
-| `SUM` | Sum numeric field |
-| `AVG` | Average numeric field |
-| `MIN` | Minimum value |
-| `MAX` | Maximum value |
-
-The client sends `groupBy[]` and `having[]` in the request to filter results.
-
----
-
-## Adjust Config
-
-Adds an `/adjust` endpoint for atomic numeric field increments/decrements.
-Prevents race conditions on stock, balance, and counter fields.
-
-```json
-"adjustConfig": {
-  "fields": ["stock_qty", "reserved_qty"],
-  "guards": [
-    {
-      "field": "stock_qty",
-      "operator": "gte",
-      "value": 0,
-      "message": "Stock cannot be negative"
-    }
-  ]
-}
-```
-
-- `fields[]` — fields that can be adjusted; must be numeric type.
-- `guards[]` — pre-condition checks; request is rejected (HTTP 422) if any
-  guard fails after applying the adjustment.
-- The client sends `{ "field": "stock_qty", "amount": -5 }` in the request.
-- Adjustment is executed as an atomic SQL UPDATE with WHERE guard.
-
----
-
-## Import Config
-
-Adds `/import-preview` and `/import-commit` endpoints for Excel (.xlsx) imports.
-
-```json
-"importConfig": {
-  "sheet": 0,
-  "startRow": 2,
-  "strategy": "upsert",
-  "upsertKey": ["sku"],
-  "columns": [
-    { "header": "SKU", "fieldName": "sku" },
-    { "header": "Product Name", "fieldName": "product_name" },
-    {
-      "header": "Category",
-      "fieldName": "category_id",
-      "lookup": { "apiPath": "/category", "matchField": "category_name", "returnField": "category_id" }
-    }
-  ]
-}
-```
-
-| Property | Notes |
-|---|---|
-| `sheet` | Sheet index (0-based) or sheet name |
-| `startRow` | First data row (1-based); default: `2` (row 1 = header) |
-| `strategy` | `"insert"` (fail on duplicate) or `"upsert"` (update on match) |
-| `upsertKey[]` | Fields used to identify existing records for upsert |
-| `columns[].header` | Excel column header text |
-| `columns[].fieldName` | RDF field to map to |
-| `columns[].lookup` | Resolve a display value to an ID before insert |
-
-Import is a two-step process: `/import-preview` validates and returns a diff;
-`/import-commit` applies changes. The client uploads the Excel file to `/import-preview`
-with a `POST multipart/form-data` request.
-
----
-
-## Processor
-
-Alternative RDF structure for custom non-CRUD endpoints. A processor payload
-does NOT have `tableName`, `fieldName`, or `action`. Each entry in `processor[]`
-defines one endpoint. Generated with `npx restforge processor create`.
-
-```json
-{
-  "description": "Sales Order custom endpoints",
-  "processor": [
-    {
-      "name": "submit-order",
-      "method": "POST",
-      "description": "Submit a draft order to pending approval",
-      "sql": {
-        "query": "UPDATE sales.sales_order SET status = 'pending_approval' WHERE so_id = $1 AND status = 'draft'",
-        "params": ["so_id"]
-      },
-      "request": {
-        "body": {
-          "so_id": { "type": "uuid", "required": true },
-          "notes": { "type": "string", "required": false, "maxLength": 200 }
-        },
-        "headers": {
-          "X-App-Code": { "type": "string", "required": true, "mapTo": "app_code" }
-        }
-      },
-      "response": {
-        "message": {
-          "success": "Sales order submitted for approval.",
-          "empty":   "Sales order not found or not in draft status.",
-          "error":   "Failed to submit sales order."
-        }
-      }
-    }
-  ]
-}
-```
-
-| Property | Required | Notes |
-|---|---|---|
-| `processor[].name` | Yes | Endpoint name — becomes file name and URL segment |
-| `processor[].method` | Yes | `GET`, `POST`, `PUT`, `PATCH`, `DELETE` |
-| `processor[].sql.query` | Conditional | Inline SQL with `$1, $2, ...` placeholders. If `sql` block present, one of `query` or `file` is required |
-| `processor[].sql.file` | Conditional | Path to external `.sql` file, relative to payload folder |
-| `processor[].sql.params` | No | Field names bound to placeholders; resolved from body/params/query/header `mapTo`; falls back to `default` if input empty |
-| `processor[].request.body` | No | Request body field schema |
-| `processor[].request.params` | No | Route params; each key adds `/:key` to the path |
-| `processor[].request.headers` | No | Header schema; use `mapTo` to rename into `input` |
-| `processor[].request.validate` | No | Default `true`. Set `false` to opt-out router-level validation |
-| `processor[].response.message` | No | `success`, `empty` (SQL mode only), `error` messages |
-| `processor[].cache.enabled` | No | Default `false`. Response cache for GET processors |
-| `processor[].cache.ttl` | No | Cache TTL in seconds; default `300` |
-
-**Field schema properties** (apply to `request.body`, `request.params`, `request.headers`):
-
-| Property | Notes |
-|---|---|
-| `type` | `string`, `number`, `integer`, `boolean`, `uuid`, `array`, `object`, `date`, `datetime` |
-| `required` | Router rejects with HTTP 400 if absent |
-| `format` | Regex whitelist: `email`, `url`, `phone-id`, `uuid` |
-| `enum` | Whitelist of allowed values |
-| `minLength` / `maxLength` | Length check for string fields |
-| `sensitive` | `true` masks value as `***MASKED***` in router debug log |
-| `default` | Fallback value for `sql.params` binding when input is empty |
-| `mapTo` | (`headers` only) field name to use in `input` object |
-
-**Generator behavior:**
-- Router (`{endpoint}.js`) — always overwritten on re-run.
-- Processor file (`processor/{endpoint}/{name}.js`) — skipped if already exists (safe to re-run).
-  Use `--force` to overwrite.
-
-**Without sql block** — payload with only `name`, `method`, and `request` is valid.
-Router registers the route with validation; processor file is generated as a manual
-implementation scaffold. Business logic is written in the processor file.
-
----
-
-## Kafka Event Publishing
-
-Publishes events to a Kafka topic after CRUD operations. Requires
-`KAFKA_ENABLED=true` in backend config.
-
-```json
-"kafka": {
-  "events": [
-    { "action": "create", "topic": "order.created.events" },
-    { "action": "update", "topic": "order.updated.events" },
-    { "action": "delete", "topic": "order.deleted.events" }
-  ]
-}
-```
-
-- `action` — CRUD action that triggers the event: `create`, `update`, `delete`,
-  `change-status`, `create-composite`, `update-composite`.
-- `topic` — Kafka topic name. Supports `{module}` and `{endpoint}` placeholders:
-  `"{module}.{endpoint}.events"`.
-- Event payload contains the full record after the operation.
-- Publishing is async and does not block the API response.
-
----
-
-## Components (Lifecycle Hooks)
-
-`components` configures CRUD lifecycle hooks that execute local JavaScript handler
-files. Added to a standard CRUD payload (one that has `tableName`).
-
-```json
-{
-  "components": [
-    {
-      "properties": {
-        "filename": "components/supplier-hooks.js",
-        "methods": [
-          {
-            "name": "validateSupplierCode",
-            "events": "onBeforeInsert",
-            "params": [
-              { "value": "{requestData}" },
-              { "value": "{user_id}" }
-            ]
-          },
-          {
-            "name": "notifySlack",
-            "events": "onAfterInsert"
-          }
-        ]
-      }
-    }
-  ]
-}
-```
-
-| Property | Required | Notes |
-|---|---|---|
-| `components[].properties.filename` | Yes | Path to handler file, relative to project root |
-| `components[].properties.methods` | Yes | List of method bindings |
-| `methods[].name` | Yes | Function name exported from the handler file |
-| `methods[].events` | Yes | Event hook (see table below) |
-| `methods[].params` | No | Template variables forwarded to the handler |
-
-**Supported event hooks:**
-
-| Event | Trigger |
-|---|---|
-| `onBeforeInsert`, `onAfterInsert` | `/create` endpoint |
-| `onBeforeUpdate`, `onAfterUpdate` | `/update` endpoint |
-| `onBeforeDelete`, `onAfterDelete` | `/delete` endpoint |
-| `onBeforeCompositeInsert`, `onAfterCompositeInsert` | `/create-composite` endpoint |
-| `onBeforeCompositeUpdate`, `onAfterCompositeUpdate` | `/update-composite` endpoint |
-
-**Template variables for `params[].value`:**
-
-| Variable | Value |
-|---|---|
-| `{tableName}` | Resource table name |
-| `{requestData}` | Full request body |
-| `{oldData}` | Data before operation (`update`, `delete`) |
-| `{newData}` | Data after operation (`create`, `update`) |
-| `{operation}` | Operation name: `insert` / `update` / `delete` |
-| `{user_id}` | User ID from request context |
-| `{timestamp}` | Execution timestamp |
-| `{record_id}` | Primary key of the affected record |
-
-**Handler file signature** (`src/components/handlers/`):
-
-```javascript
-async function handlerName(/* resolved params... */, services) {
-  const { db, logger, redis, kafka, cache } = services;
-  // business logic
-  return { success: true, message: '...' };
-}
-module.exports = { handlerName };
-```
-
-- `services` is injected automatically as the last argument; no need to declare it
-  in `params[]`.
-- All events are **blocking** — `return { success: false }` or throwing an exception
-  rolls back the entire transaction.
-- If `components` is absent from the payload, CRUD operates normally without hooks.
+# Reference: RDF Advanced Features
+
+> **Offline mirror.** This file mirrors the RDF catalog of the installed
+> RESTForge platform (`restforge-handbook/catalogs/rdf/`) and the key shapes the
+> runtime actually reads. The live platform is authoritative — when this file
+> and the platform disagree, trust the platform, then update this file. For
+> `fieldValidation` constraints, see `references/field-validation.md`.
+
+This reference covers advanced RDF payload features beyond standard CRUD fields.
+
+**Grounding tools for this file.** Reading this reference is not grounding — it
+explains the shapes, the tools return what the *installed* platform accepts:
+
+| Editing | Call first |
+|---|---|
+| `fieldValidation` on master or detail columns | `codegen_get_field_validation_catalog` |
+| `datatablesQuery`, `viewQuery`, `viewName`, `exportQuery`, `detailQuery`, and `file:` query references | `codegen_get_query_declarative_catalog` |
+| Any SELECT / WITH statement before it is pasted into the payload or a `.sql` file | `codegen_validate_sql` (live EXPLAIN, executes no rows) |
+
+Whatever the section, the finished payload goes through `codegen_validate_payload`
+before `codegen_create_endpoint` — a processor payload through
+`codegen_create_processor`.
+
+**Start from a generated payload.** `codegen_generate_payload` writes `tableName`,
+`primaryKey`, `fieldName`, `action`, `fieldValidation`, `uniqueConstraints`,
+`dateTimeFields`, `deleteReferences`, `softDelete`, and (for a table with an
+`is_active` column) `defaultScope`. Edit that file; do not write those keys by
+hand. Later `codegen_generate_payload` / `codegen_sync_payload` runs keep the
+customisations made to those keys (see SKILL.md § RDF Payload).
+
+---
+
+## Table of Contents
+
+1. [The `action` Block](#the-action-block)
+2. [Data Source Resolution](#data-source-resolution)
+3. [Query File Reference](#query-file-reference)
+4. [Field Lookup](#field-lookup)
+5. [Default Scope](#default-scope)
+6. [Workflow (Change-Status)](#workflow-change-status)
+7. [Master-Detail (Composite)](#master-detail-composite)
+8. [Aggregate Config](#aggregate-config)
+9. [Adjust Config](#adjust-config)
+10. [Import Config](#import-config)
+11. [Processor](#processor)
+12. [Kafka Event Publishing](#kafka-event-publishing)
+13. [Components (Lifecycle Hooks)](#components-lifecycle-hooks)
+14. [Other RDF Blocks](#other-rdf-blocks)
+
+---
+
+## The `action` Block
+
+`action` is required. Every key is a boolean flag; an unknown key produces a
+warning, a non-boolean value is an error. A feature block alone does not create
+its endpoint — the matching flag must be `true`.
+
+| Flag | Endpoint(s) | Also needs |
+|---|---|---|
+| `datatables` | `POST /datatables` | — |
+| `create`, `update`, `delete` | `POST /create`, `/update`, `/delete` | — |
+| `first`, `read`, `lookup` | `POST /first`, `/read`, `GET`/`POST /lookup` | — |
+| `export` | `/export` (Excel) | — |
+| `import` | `/import-upload`, `/import-preview`, `/import-commit`, `/import-status` | `importConfig` with `enabled: true` |
+| `upload` | file upload routes | `uploadConfig` |
+| `adjust` | `POST /adjust` | `adjustConfig` |
+| `aggregate` | `POST /aggregate` | `aggregateConfig` only when JOINs are needed |
+| `workflow` | `POST /change-status` | `workflow` block |
+| `createComposite`, `updateComposite`, `readComposite` | `/create-composite`, `/update-composite`, `/read-composite` | `masterDetail` block |
+| `restore` | `/restore` | `softDelete.enabled: true` (schema-derived) |
+
+`/export` is registered for every CRUD module whether or not the flag is set;
+the flag only shows up in `GET /info`. `/import-*` is registered only when
+`importConfig.enabled` is `true`; set `action.import: true` as well so the RDF
+and `/info` describe the endpoints that actually exist.
+
+---
+
+## Data Source Resolution
+
+RESTForge resolves data sources per endpoint using a priority chain. Define
+only what is needed; the platform falls back automatically.
+
+| Endpoint | Resolution order |
+|---|---|
+| `/datatables` | `datatablesQuery` → `SELECT * FROM` (`viewName` or `tableName`) |
+| `/read`, `/first`, `/lookup` | `viewName` → `viewQuery` → `tableName` |
+| `/export` | `exportQuery` → `SELECT {fieldName} FROM {tableName}` |
+| `/read-composite` (detail) | `masterDetail.detailConfig.detailQuery` → `SELECT * FROM {detailTable} WHERE {foreignKey} = ? ORDER BY {line_number or detail primaryKey}` |
+
+**`viewName`** — reference a database VIEW:
+```json
+"viewName": "v_order_summary"
+```
+
+**`viewQuery`** — inline SQL (virtual view, no DB object created). It is wrapped
+as a subquery, so JOINed columns can be filtered in WHERE:
+```json
+"viewQuery": "SELECT o.*, c.customer_name FROM orders o JOIN customers c ON o.customer_id = c.customer_id"
+```
+
+**`datatablesQuery`** — the base SELECT for the paginated list. The runtime wraps
+it with search, sort, and paging; do not add those clauses yourself:
+```json
+"datatablesQuery": "SELECT o.*, c.customer_name FROM orders o JOIN customers c ON o.customer_id = c.customer_id"
+```
+
+**`datatablesWhere`** — the whitelist of columns the `/datatables` `searchBy`
+parameter may target; add `"all"` to allow cross-column search. Entries use the
+column name as it appears in the SELECT result, never with a table alias
+(`a.supplier_code` is rejected). Every entry other than `all` must exist in the
+columns `/datatables` returns; `codegen_create_endpoint` checks this. A
+`searchBy` value outside the list is rejected with HTTP 400.
+```json
+"datatablesWhere": ["supplier_code", "supplier_name", "all"]
+```
+
+Write source is always `tableName` — `viewName`/`viewQuery` are read-only.
+
+**JOIN columns in the list.** To show columns of referenced tables
+(`supplier_name` next to `supplier_id`), prefer `codegen_sync_payload` with
+`expandFk` over a hand-written JOIN: it writes `query/<table>-join.sql` and
+points `datatablesQuery` (and, with `expandFk: "both"`, `viewQuery`) at it.
+
+Ground every one of these keys with `codegen_get_query_declarative_catalog`
+before writing them, and run the SQL through `codegen_validate_sql` first: a
+JOIN or column typo here surfaces as a runtime 500 on `/datatables` or `/export`,
+not as a payload validation error.
+
+---
+
+## Query File Reference
+
+SQL queries can be stored in external `.sql` files using the `file:` prefix.
+The path is relative to the `payload/` folder.
+
+```json
+"datatablesQuery": "file:query/orders-datatables.sql",
+"exportQuery": "file:query/orders-export.sql"
+```
+
+Convention for folder structure:
+```
+payload/
+├── order.json
+└── query/
+    ├── orders-datatables.sql
+    └── orders-export.sql
+```
+
+`viewName` does not accept `file:` because it names a database object. For
+master-detail, `detailQuery` lives under `masterDetail.detailConfig` and may use
+`file:` too.
+
+The `file:` form hides the SQL from a quick payload review, which makes
+`codegen_validate_sql` on the file content worth more here than for inline
+queries.
+
+---
+
+## Field Lookup
+
+`fieldNameLookup` (root level, optional) selects which columns `/lookup` returns
+as `id` and `text`. There is no per-field lookup key in RDF; a dropdown on the
+frontend is a UDF `dataSource` that calls this endpoint.
+
+```json
+"fieldNameLookup": {
+  "id": "category_id",
+  "text": "category_code||' - '||category_name as display_text"
+}
+```
+
+| Key | Notes |
+|---|---|
+| `id` | Column returned as `id` |
+| `text` | Column **or SQL expression** returned as `text`. PostgreSQL `||` concatenation is translated to `CONCAT()` for MySQL automatically |
+
+- Both keys are required once the block exists.
+- Without the block, the text column is auto-detected by name (`name`, `code`,
+  `title`, `text`, `label`, `tag`).
+- With the block, `/lookup` search runs on the columns extracted from `text`.
+
+---
+
+## Default Scope
+
+Automatic WHERE filter injected on the `lookup` and `read` actions only.
+`/datatables` and `/first` are not affected.
+
+```json
+"defaultScope": {
+  "lookup": { "is_active": true },
+  "read": { "is_active": true }
+}
+```
+
+- The keys are action names: only `lookup` (`GET` and `POST /lookup`) and `read`
+  (`POST /read`). Any other key produces a warning and is ignored.
+- Each action maps `column: value`. The column must be in `fieldName`; the value
+  must be a boolean, string, or number literal. Values are not taken from the
+  request or the JWT.
+- The filter is combined with the user-supplied WHERE via AND.
+- For an `is_active` column, `codegen_generate_payload` writes the block
+  automatically and `codegen_sync_payload` keeps only the `is_active` key in
+  step with the table; custom keys (e.g. `show_in_store`) are left alone.
+
+---
+
+## Workflow (Change-Status)
+
+Adds a `POST /change-status` endpoint with state machine validation. Needs both
+`action.workflow: true` and the `workflow` block.
+
+```json
+"action": { "workflow": true },
+"workflow": {
+  "statusField": "status",
+  "transitions": {
+    "draft":     ["confirmed", "cancelled"],
+    "confirmed": ["closed", "cancelled"],
+    "closed":    [],
+    "cancelled": []
+  },
+  "hooks": {
+    "confirmed": {
+      "onBefore": [],
+      "onAfter": [
+        {
+          "type": "api",
+          "method": "POST",
+          "url": "/stock-management/process-inbound",
+          "body": { "stock_inbound_id": "{{id}}", "status": "{{newStatus}}", "previous_status": "{{oldStatus}}" },
+          "blocking": true,
+          "timeout": 10000
+        }
+      ]
+    }
+  }
+}
+```
+
+| Property | Notes |
+|---|---|
+| `statusField` | Column that holds the status (default `"status"`); must be in `fieldName` |
+| `transitions` | Map `current status → [allowed target statuses]`. `null`/absent = any transition allowed. A target outside the list → HTTP 422 `Status transition not allowed` |
+| `hooks.<target status>` | Keyed by the **target** status, not by an action name. Holds `onBefore[]` and/or `onAfter[]` |
+| hook `type` | Only `"api"` |
+| hook `method` | `POST` (default), `PUT`, or `PATCH` |
+| hook `url` | Full URL, absolute path (`/api/...`), or short path (`/{endpoint}/{action}`) |
+| hook `body` / `headers` | Template variables `{{id}}`, `{{newStatus}}`, `{{oldStatus}}`, `{{record.<field>}}` |
+| hook `blocking` | `true` = a failed call rolls back the transaction and the request is answered HTTP 502; `false` (default) = fire-and-forget |
+| hook `timeout` | Milliseconds, default `10000` |
+
+The request body carries the primary key (or `id`) and the target `status`.
+`onBefore` runs before the UPDATE, `onAfter` after it but before COMMIT. Local
+JavaScript hooks for the same operation use the `onBeforeWorkflow` /
+`onAfterWorkflow` component events (see Components). The frontend buttons are
+UDF `workflowActions`, a separate file — never put `workflowActions` in the RDF.
+
+---
+
+## Master-Detail (Composite)
+
+Adds `/create-composite`, `/update-composite`, and `/read-composite`. Needs the
+three composite flags in `action` and a `masterDetail` block. Generate the block
+instead of writing it: `codegen_generate_payload` with `detail: "<detail table>"`
+fills `masterDetail` and `detailConfig` (including `fieldValidation` and
+`foreignKeys`) from the database and writes the detail query file.
+
+```json
+"action": { "createComposite": true, "updateComposite": true, "readComposite": true },
+"masterDetail": {
+  "enabled": true,
+  "detailTable": "stock_inbound_item",
+  "foreignKey": "stock_inbound_id",
+  "cascadeDelete": true,
+  "transactionMode": "required",
+  "detailConfig": {
+    "tableName": "stock_inbound_item",
+    "primaryKey": "stock_inbound_item_id",
+    "fieldName": ["stock_inbound_item_id", "stock_inbound_id", "line_number", "item_product_id", "qty_received", "unit_price", "total_amount"],
+    "detailQuery": "file:query/stock-inbound-detail.sql",
+    "requiredFields": ["line_number", "item_product_id", "qty_received", "unit_price"],
+    "autoCalculateFields": {
+      "total_amount": { "type": "calculated", "formula": "qty_received * unit_price" }
+    }
+  },
+  "headerCalculations": {
+    "total_items":  { "type": "count", "source": "items.length" },
+    "total_amount": { "type": "sum",   "source": "items.total_amount" }
+  }
+}
+```
+
+| Property | Notes |
+|---|---|
+| `enabled` | `true` activates the feature |
+| `detailTable` / `foreignKey` | Detail table and its FK column to the header |
+| `cascadeDelete` | `true` = deleting the header first deletes its detail rows in the same transaction (all four dialects; no `ON DELETE CASCADE` needed) |
+| `transactionMode` | Only `"required"` |
+| `detailConfig.tableName`, `primaryKey`, `fieldName` | Required. Every listed column must exist in the detail table; `codegen_create_endpoint` checks this against the database |
+| `detailConfig.detailQuery` | Optional SELECT for detail rows; inline or `file:` |
+| `detailConfig.requiredFields` | Columns required on each detail insert |
+| `detailConfig.autoCalculateFields` | Per-row values: `calculated` (formula `colA * colB`, computed by the app) or `generated` (DB GENERATED column, excluded from SQL) |
+| `detailConfig.fieldValidation`, `foreignKeys` | Filled by generate; consumed by `codegen_migrate_payload` to build the UDF `details[]` |
+| `headerCalculations` | Header columns computed from detail rows: `type` = `count`, `sum`, `avg`, `min`, `max`; `source` = `items.length` for count, `items.<detail column>` otherwise. The key must be a header column in root `fieldName` |
+
+Fill `headerCalculations` and `calculated` formulas by hand after generating —
+they are business decisions. `autoCalculateFields` goes **inside**
+`detailConfig`; placing it next to `detailConfig` is rejected. The `_manualStub`
+key written by generate is documentation only and can be removed.
+
+At runtime a detail row field outside the detail table columns is rejected with
+400 and the whole request is rolled back. `/update-composite` with only detail
+operations is valid and recomputes `headerCalculations`.
+
+---
+
+## Aggregate Config
+
+`/aggregate` runs COUNT, SUM, AVG, MIN, and MAX over the resource. It needs
+`action.aggregate: true`. The **request body** carries the operations; the RDF
+only declares the JOINs a request may use, so clients cannot join arbitrary
+tables.
+
+```json
+"action": { "aggregate": true },
+"aggregateConfig": {
+  "joins": {
+    "warehouse": {
+      "tableName": "warehouse",
+      "joinType": "LEFT",
+      "sourceField": "warehouse_id",
+      "targetField": "warehouse_id",
+      "fields": ["warehouse_code", "warehouse_name"]
+    }
+  }
+}
+```
+
+| `joins.<name>` key | Notes |
+|---|---|
+| *(the key)* | Join name referenced by the request (`"joins": ["warehouse"]`) |
+| `tableName` | Joined table |
+| `joinType` | `INNER`, `LEFT`, or `RIGHT` (the runtime rejects `FULL`) |
+| `sourceField` / `targetField` | Column on the main table / on the joined table |
+| `fields` | Joined-table columns the request may use in `group_by` and `where` |
+
+Request body:
+
+```json
+{
+  "joins": ["warehouse"],
+  "operations": [
+    { "function": "sum", "field": "stock_qty", "alias": "total_stock" },
+    { "function": "count", "field": "*", "alias": "total_items" }
+  ],
+  "group_by": ["warehouse_name"],
+  "where": { "logic": "AND", "conditions": [{ "key": "is_active", "operator": "=", "value": true }] },
+  "having": [{ "function": "sum", "field": "stock_qty", "operator": ">", "value": 0 }]
+}
+```
+
+- Functions: `count`, `sum`, `avg`, `min`, `max` (case-insensitive). `field: "*"`
+  is allowed for `count` only.
+- `where` uses the same structure as `/read`: `{ logic, conditions: [{ key,
+  operator, value }] }`, nested groups allowed; a key outside the readable and
+  join fields is rejected with 400. `having` operators: `=`, `<>`, `!=`, `>`,
+  `<`, `>=`, `<=`.
+- Main-table fields must be readable fields of the resource; an alias is
+  alphanumeric with underscores.
+- Without `operations` the endpoint returns `{ "count": N }`.
+- Without `group_by` the response data is one object; with `group_by` it is an
+  array of rows.
+
+For a chart or KPI built from several tables, a backend dashboard
+(`codegen_create_dashboard`, SQL widgets) is usually the better fit than
+`/aggregate` per resource.
+
+---
+
+## Adjust Config
+
+Adds `POST /adjust` for atomic increments/decrements on numeric columns
+(stock, balance, counters). Needs `action.adjust: true` and `adjustConfig`.
+
+```json
+"action": { "adjust": true },
+"adjustConfig": {
+  "fields": {
+    "stock": { "type": "number", "min": 0, "allowNegativeResult": false }
+  },
+  "reasonRequired": true
+}
+```
+
+| Property | Notes |
+|---|---|
+| `fields.<column>` | Column that may be adjusted. Must be a physical column of the table |
+| `fields.<column>.type` | Must be `"number"` (default) |
+| `fields.<column>.min` | Lower bound after the adjustment, default `0` |
+| `fields.<column>.allowNegativeResult` | `false` = the UPDATE carries a guard `column + value >= min`; a violation is answered HTTP 409. Default `true` (no guard) |
+| `reasonRequired` | `true` = the request must include a non-empty `reason` |
+
+Request body:
+
+```json
+{
+  "product_id": "018f...",
+  "adjustments": [{ "field": "stock", "value": -5 }],
+  "reason": "Damaged goods"
+}
+```
+
+`value` is a non-zero number. A column not listed in `fields`, a missing primary
+key, or an empty `adjustments` array is answered HTTP 400. The adjust call fires
+the `onBeforeAdjust` / `onAfterAdjust` component events.
+
+The guard is enforced at adjust time only. A bound that must hold for every
+write path belongs in `fieldValidation` as well — check what the installed
+platform offers with `codegen_get_field_validation_catalog`.
+
+---
+
+## Import Config
+
+Excel (.xlsx) import through four endpoints: `POST /import-upload`,
+`POST /import-preview`, `POST /import-commit`, `GET /import-status`. The routes
+exist only when `importConfig.enabled` is `true`; set `action.import: true` too.
+
+```json
+"action": { "import": true },
+"importConfig": {
+  "enabled": true,
+  "upsertKeys": ["supplier_code"],
+  "upsertStrategy": "update_existing",
+  "requiredFields": ["supplier_code", "supplier_name"],
+  "maxFileSize": "10MB",
+  "allowedFormats": ["xlsx"],
+  "chunkSize": 100,
+  "lookupFields": {
+    "city_id": {
+      "targetField": "city_id",
+      "lookupTable": "city",
+      "lookupColumn": "city_name",
+      "lookupIdColumn": "city_id",
+      "required": true
+    }
+  }
+}
+```
+
+| Property | Notes |
+|---|---|
+| `enabled` | Must be `true` to register the routes |
+| `upsertKeys` | Columns that identify an existing row (default: the primary key) |
+| `upsertStrategy` | `update_existing` (default), `insert_only`, or `skip_existing` |
+| `requiredFields` | Columns that must be filled in every imported row |
+| `maxFileSize` | Upload size limit, e.g. `"10MB"` |
+| `allowedFormats` | Default `["xlsx"]` |
+| `chunkSize` | Rows per INSERT/UPDATE batch, default `100` |
+| `lookupFields.<name>` | Resolve a display value in the sheet to an ID: `targetField` (resource column), `lookupTable`, `lookupColumn` (value shown in Excel), `lookupIdColumn` (value stored), `required` (`true` = reject the row when no match) |
+
+Column headers and formats follow `columnFormats` and the field labels of the
+module, the same source `/export` uses, so an exported file can be re-imported.
+`fieldValidation` applies to `/create` and `/update` only, not to import; use
+`requiredFields` and `lookupFields.required` for import-time checks.
+
+---
+
+## Processor
+
+Alternative RDF structure for custom non-CRUD endpoints. A processor payload
+does NOT have `tableName`, `fieldName`, or `action`. Each entry in `processor[]`
+defines one endpoint. Generated with `npx restforge processor create`.
+
+```json
+{
+  "description": "Sales Order custom endpoints",
+  "processor": [
+    {
+      "name": "submit-order",
+      "method": "POST",
+      "description": "Submit a draft order to pending approval",
+      "sql": {
+        "query": "UPDATE sales.sales_order SET status = 'pending_approval' WHERE so_id = $1 AND status = 'draft'",
+        "params": ["so_id"]
+      },
+      "request": {
+        "body": {
+          "so_id": { "type": "uuid", "required": true },
+          "notes": { "type": "string", "required": false, "maxLength": 200 }
+        },
+        "headers": {
+          "X-App-Code": { "type": "string", "required": true, "mapTo": "app_code" }
+        }
+      },
+      "response": {
+        "message": {
+          "success": "Sales order submitted for approval.",
+          "empty":   "Sales order not found or not in draft status.",
+          "error":   "Failed to submit sales order."
+        }
+      }
+    }
+  ]
+}
+```
+
+| Property | Required | Notes |
+|---|---|---|
+| `processor[].name` | Yes | Endpoint name — becomes file name and URL segment |
+| `processor[].method` | Yes | `GET`, `POST`, `PUT`, `PATCH`, `DELETE` |
+| `processor[].sql.query` | Conditional | Inline SQL with `$1, $2, ...` placeholders. If `sql` block present, one of `query` or `file` is required |
+| `processor[].sql.file` | Conditional | Path to external `.sql` file, relative to payload folder |
+| `processor[].sql.params` | No | Field names bound to placeholders; resolved from body/params/query/header `mapTo`; falls back to `default` if input empty |
+| `processor[].request.body` | No | Request body field schema |
+| `processor[].request.params` | No | Route params; each key adds `/:key` to the path |
+| `processor[].request.headers` | No | Header schema; use `mapTo` to rename into `input` |
+| `processor[].request.validate` | No | Default `true`. Set `false` to opt-out router-level validation |
+| `processor[].response.message` | No | `success`, `empty` (SQL mode only), `error` messages |
+| `processor[].cache.enabled` | No | Default `false`. Response cache for GET processors |
+| `processor[].cache.ttl` | No | Cache TTL in seconds; default `300` |
+
+**Field schema properties** (apply to `request.body`, `request.params`, `request.headers`):
+
+| Property | Notes |
+|---|---|
+| `type` | `string`, `number`, `integer`, `boolean`, `uuid`, `array`, `object`, `date`, `datetime` |
+| `required` | Router rejects with HTTP 400 if absent |
+| `format` | Regex whitelist: `email`, `url`, `phone-id`, `uuid` |
+| `enum` | Whitelist of allowed values |
+| `minLength` / `maxLength` | Length check for string fields |
+| `sensitive` | `true` masks value as `***MASKED***` in router debug log |
+| `default` | Fallback value for `sql.params` binding when input is empty |
+| `mapTo` | (`headers` only) field name to use in `input` object |
+
+**Generator behavior:**
+- Router (`{endpoint}.js`) — always overwritten on re-run.
+- Processor file (`processor/{endpoint}/{name}.js`) — skipped if already exists (safe to re-run).
+  Use `--force` to overwrite.
+
+**Without sql block** — payload with only `name`, `method`, and `request` is valid.
+Router registers the route with validation; processor file is generated as a manual
+implementation scaffold. Business logic is written in the processor file.
+
+`sql.query` is executed as written: run it through `codegen_validate_sql` first
+when it is a SELECT, and keep in mind that `codegen_create_processor` takes the
+payload as a bare file name (no path form), unlike the dashboard generators.
+
+---
+
+## Kafka Event Publishing
+
+Publishes an event to one Kafka topic after insert, update, or delete. Requires
+`KAFKA_ENABLED=true` in the backend config as well.
+
+```json
+"kafka": {
+  "enabled": true,
+  "topic": "inventory.stock_inbound",
+  "keyField": "stock_inbound_id",
+  "publishOn": { "insert": true, "update": true, "delete": false }
+}
+```
+
+| Property | Notes |
+|---|---|
+| `enabled` | `true` activates publishing for this resource |
+| `topic` | Target topic name |
+| `keyField` | Record field used as the message key (default: the primary key) |
+| `publishOn.insert` / `update` / `delete` | Which operations publish |
+
+There is one topic per resource and no per-action topic list. To consume events,
+generate a consumer with `codegen_create_kafka_consumer`.
+
+---
+
+## Components (Lifecycle Hooks)
+
+`components` binds CRUD lifecycle events to functions in local JavaScript
+handler files. Added to a standard CRUD payload (one that has `tableName`).
+
+```json
+{
+  "components": [
+    {
+      "properties": {
+        "filename": "components/supplier-hooks.js",
+        "methods": [
+          {
+            "name": "validateSupplierCode",
+            "events": "onBeforeInsert",
+            "params": [
+              { "value": "{requestData}" },
+              { "value": "{user_id}" }
+            ]
+          },
+          {
+            "name": "notifySlack",
+            "events": "onAfterInsert",
+            "params": [{ "value": "{newData}" }]
+          }
+        ]
+      }
+    }
+  ]
+}
+```
+
+| Property | Required | Notes |
+|---|---|---|
+| `components[].properties.filename` | Yes | Handler path **relative to `src/`**: `components/supplier-hooks.js` loads `src/components/supplier-hooks.js`. `..`, absolute paths, and characters outside `[a-zA-Z0-9._/-]` are rejected |
+| `components[].properties.methods` | Yes | List of method bindings |
+| `methods[].name` | Yes | Function name exported from the handler file |
+| `methods[].events` | Yes | Event hook (see table below) |
+| `methods[].params` | Yes in practice | Template variables forwarded to the handler, in order. Always write the array (use `[]` for none): a binding without `params` fails when the event fires |
+
+**Supported event hooks:**
+
+| Event | Trigger |
+|---|---|
+| `onBeforeInsert`, `onAfterInsert` | `/create` endpoint |
+| `onBeforeUpdate`, `onAfterUpdate` | `/update` endpoint |
+| `onBeforeDelete`, `onAfterDelete` | `/delete` endpoint |
+| `onBeforeCompositeInsert`, `onAfterCompositeInsert` | `/create-composite` endpoint |
+| `onBeforeCompositeUpdate`, `onAfterCompositeUpdate` | `/update-composite` endpoint |
+| `onBeforeAdjust`, `onAfterAdjust` | `/adjust` endpoint |
+| `onBeforeWorkflow`, `onAfterWorkflow` | `/change-status` endpoint |
+
+**Template variables for `params[].value`:**
+
+| Variable | Value |
+|---|---|
+| `{tableName}` | Resource table name |
+| `{requestData}` | Full request body |
+| `{oldData}` | Data before operation (`update`, `delete`) |
+| `{newData}` | Data after operation (`create`, `update`) |
+| `{operation}` | Operation name: `insert` / `update` / `delete` |
+| `{user_id}` | User ID from request context |
+| `{timestamp}` | Execution timestamp |
+| `{record_id}` | Primary key of the affected record |
+
+A value that is exactly one variable passes the raw value (object, array);
+a string mixing text and variables is interpolated.
+
+**Handler file signature** (e.g. `src/components/supplier-hooks.js`):
+
+```javascript
+async function validateSupplierCode(requestData, userId, services) {
+  const { db, logger, redis, kafka, cache } = services;
+  if (!/^[A-Z]{3}\d{3}$/.test(requestData.supplier_code)) {
+    return { success: false, message: 'Supplier code must look like ABC123' };
+  }
+  return { success: true };
+}
+module.exports = { validateSupplierCode };
+```
+
+- `services` is injected automatically as the last argument; no need to declare
+  it in `params[]`. It also carries `idempotency`, `idgen`, `storage`,
+  `websocket`, `createResponse`, `createError`, and `createValidationError`.
+- All events are **blocking**, including the `onAfter*` ones. Returning
+  `{ success: false, message }` or throwing rolls back the whole transaction.
+- A rejection is answered **HTTP 400** with the hook's own `message` only; the
+  function name and file path are written to the server log, not to the client.
+- If `components` is absent from the payload, CRUD operates normally without hooks.
+
+A lifecycle hook is not a substitute for declared validation: keep rule checks
+that `fieldValidation` can express in the payload, grounded with
+`codegen_get_field_validation_catalog`, and reserve `components` for logic the
+catalog genuinely cannot express (external calls, cross-table effects,
+notifications, status locks).
+
+---
+
+## Other RDF Blocks
+
+These blocks are read by the payload validator or the generator. Most are
+schema-derived and written by `codegen_generate_payload`; the handbook pages
+under `restforge-handbook/catalogs/rdf/` hold the full rules.
+
+| Block | Purpose | Written by |
+|---|---|---|
+| `dateTimeFields` | Per-column `type` (`date`, `timestamp`, `timestamptz`, `time`) so the runtime normalises input and formats output. `format` is allowed for `time` only; `date`/`timestamp` always follow `DATEFORMAT`/`DATETIMEFORMAT` | generate |
+| `uniqueConstraints` | `{name, fields}` from the database; names the conflicting field in a 409 response | generate / sync |
+| `deleteReferences` | Child tables that block a delete (`{table, column, references}`); used in the 409 response of `/delete` | generate |
+| `softDelete` (+ `softDeleteFk*` metadata) | Soft-delete behaviour derived from the SDF. Only `visibility` (`active_only`, `deleted_only`, `include_deleted`) is meant to be edited | generate |
+| `auditColumns` | `false`/`null` disables, object renames the four audit columns | manual override |
+| `concurrency` | Optimistic concurrency for `update`/`update-composite`: `{versionColumn, compare}`; `compare` = `version` (integer column, default) or `timestamp` (PostgreSQL only). Pairs with UDF `versionField` | manual |
+| `fieldPolicy` | Per-column `strategies`: `lock` (SELECT ... FOR UPDATE) and/or `audit` (writes `<table>_audit`); `"*": {"strategies": ["audit"]}` audits every column. Replaces `fieldProtection` | manual |
+| `uploadConfig` | File fields (JSON columns) with `maxFiles`, `maxFileSize`, `allowedTypes`, `allowedMimeTypes`, `storagePrefix`; needs `action.upload: true` | manual |
+| `authGuard` | `{enabled, appCode, publicPaths}`: JWT verification and per-endpoint permission; generates `src/plugins/<project>-auth-guard.js` | manual |
+| `columnFormats` | Excel column formats shared by `/export` and import | manual |