@astram/guide 0.2.25 → 0.2.26

package/CAPABILITIES.md

@@ -331,7 +331,26 @@ imports: [
   with the call's `batchId` and the row's `batchIndex` (ADR-0039). Inside `transaction(fn)` it
   joins the outer transaction.
 - `transaction(fn)` runs several pipe calls in one unit of work (rolls back on any failed write);
-  `lockForUpdate(typeKey, id)` serialises a check-then-write.
+  `lockForUpdate(typeKey, id)` serialises a check-then-write: lock first, then re-read and
+  re-check, then write. It locks only a row you could `update` (your tenant, your `readScope`
+  subtree); a foreign or out-of-scope row is never locked.
+  - **Behaviour change (engine 0.19):** before this version a missing, foreign or out-of-scope id
+    was silently not locked. From this version `lockForUpdate` on such an id throws the same deny
+    an `update` of that id gets (`AuthorizationDeniedError`, reason `no_record_permit`; one answer
+    for all three) and locks nothing. Code that locks first and checks for `null` after must
+    either check existence first (`get` inside the transaction, then `lockForUpdate`) or catch
+    that deny. Prefer catching the deny when the re-check after the lock must see the latest
+    committed state: a plain read before the lock fixes the transaction's read snapshot before
+    the lock wait. The deny comes before any lock statement, so a caught deny leaves the
+    transaction usable: you may continue and commit. An `EngineUnavailableError` from it (the engine's
+    lock-probe pool did not answer within 2000 ms) is different: let it propagate (503); no lock
+    was taken.
+  - A row another transaction inserted but has not committed yet is "not found" (it used to wait
+    for it). If a row is deleted between the existence check and the lock, the lock briefly holds
+    the empty key range around that id until your transaction ends (nothing is revealed).
+  - **Read lock pattern:** to hold rows you only read, call `tx.authorize(typeKey, id, 'read')`
+    inside `transaction(fn)` (a locking read of the row as you may see it), taking ids in sorted
+    order to avoid deadlocks. Like `lockForUpdate`, it never locks a foreign or out-of-scope row.
 - `@Field({ immutable: true })` create-only fields; `@Field({ nullable: true })` fields that
   create, update and `createMany` may set to NULL (`json` included: SQL NULL, not `"null"`;
   the audit row lists the cleared keys, never values; the column must be NULL-able in your
@@ -629,7 +648,8 @@ upload(@Req() req: Record<string, unknown>) {
   only lower it. A route naming a profile the app did not register aborts boot. Needs
   `@nestjs/platform-express` (an optional peer; the scaffold has it).
 - **Profiles** — turn them on with `ShieldModule.forRoot({ fileProfiles: ['xlsx', 'csv', 'pdf', 'docx'] })`
-  (default `['xlsx']`; an unknown, repeated or empty list aborts boot). You cannot define your own.
+  (default `['xlsx']`; an unknown, repeated or empty list aborts boot). For a format of your own, see
+  **App profiles** below.
   | Profile | Default cap | Declared types accepted (stored as the first) | Content check |
   |---|---|---|---|
   | `xlsx` | 10 MiB | spreadsheetml `.sheet` | ZIP + `[Content_Types].xml` names a spreadsheetml main part |
@@ -653,6 +673,46 @@ upload(@Req() req: Record<string, unknown>) {
   `xl/embeddings/` (a Word chart keeps its data there); they count toward the limits and are never
   opened. Files Office, LibreOffice, Google and common zip libraries write pass; a file with bytes
   added after the zip's end record (some mail gateways pad attachments) is refused.
+- **App profiles (`appFileProfiles`, DP1)** — a format you own (a vendor export, a `.json` record,
+  a domain CSV dialect) is registered next to the shipped profiles and goes through the same
+  intake (cap, type, extension, your `magic`, sha256, tenant key) and `FilesService`:
+  ```ts
+  ShieldModule.forRoot({
+    storage: 'env',
+    fileProfiles: ['xlsx'],
+    appFileProfiles: [
+      {
+        name: 'inspection_json',
+        profile: {
+          maxBytes: 2 * 1024 * 1024,
+          contentTypes: ['application/json'], // first = the stored type
+          extensions: ['.json'],
+          magic: isInspectionJson, // your content check over the raw bytes; false or a throw = 415
+        },
+      },
+    ],
+  });
+  // route: @UseInterceptors(FileIntake({ field: 'file', profile: 'inspection_json' }))
+  ```
+  Boot refuses (naming the entry index and rule): more than 16 entries; a name not
+  `^[a-z][a-z0-9_]{0,31}$` or equal to a shipped name or repeated; `magic` not a function;
+  `maxBytes` not a positive integer ≤ 50 MiB; a content type that is not a plain lower-case
+  `type/subtype` (no parameters; stricter than the spec, so a malformed value can never reach a
+  download header), that is denied, or that a shipped profile (registered or not) or another app
+  profile already uses; an extension not `^\.[a-z0-9]{1,10}$` or already used. **Denied types**
+  (`APP_CONTENT_TYPE_DENY_LIST` + `isDeniedAppContentType` in `@astram/guard`; wider than the spec):
+  `text/html`, `application/octet-stream`; xml (`application/xml`, `text/xml`, and any `+xml`
+  subtype such as `image/svg+xml`, `application/xhtml+xml`); php; `application/wasm`; shell and
+  script types (`sh`, `csh`, `bash`, `powershell`, python, perl, ruby, `bat`, DOS programs); and
+  every javascript / ecmascript / jscript variant. **Cap:** your profile's `maxBytes` is its
+  ceiling. An explicit `OBJECT_STORAGE_MAX_BYTES` can only **lower** it, never raise it (stricter
+  than the spec; shipped profiles differ: the env value replaces their cap). Downloads serve the
+  app type with the same `attachment` + `nosniff` headers.
+  **ZIP-based formats:** your `magic` MUST call `zipWithinLimits(bytes)` (from `@astram/core`)
+  before reading anything else; it applies every archive limit above without inflating anything.
+  For another OOXML format, `ooxmlMagic('<main part content type>')` is the shipped xlsx/docx check.
+  The framework never parses app-profile bytes and the file analyser never receives them (reading
+  them is DP2 or your own isolated worker).
 - **`FilesService`** (provided by `IdentityFeatureModule`) — `store(intake)`, `open(uuid)`,
   `remove(uuid)`. No actor argument: tenant, owner and audit actor come from the session. `store`
   puts the object at `t/<tenant>/<uuid>`, then creates the record (gate, tenant stamp, audit); a
@@ -729,10 +789,11 @@ upload(@Req() req: Record<string, unknown>) {
 - **Set `POOL_MAX_ENGINE`** (e.g. 10). Unset, the engine pool uses `APP_DB_MAX_CONNECTIONS`, and
   the single-layout budget `workers × (POOL_MAX_CORE + POOL_MAX_ENGINE + APP_AUTHZ_POOL_MAX) +
   headroom` fails boot. `APP_DB_MAX_USER_CONNECTIONS` (the account's cap, if the host sets one) is
-  checked too. The budget also counts 1 connection per worker for the audit sealer's own pool.
+  checked too. The budget also counts 1 connection per worker for the audit sealer's own pool and
+  2 per worker for the engine's lock-probe pool.
 - **Set `POOL_WORKERS` and `APP_DB_MAX_CONNECTIONS` too** (your real worker count and what the
   server gives you). Their defaults (8 workers, 250) describe a dedicated four-database server; on
-  them the single-layout budget is 8 × 27 + 40 = 256 > 250 and boot fails.
+  them the single-layout budget is 8 × 29 + 40 = 272 > 250 and boot fails.
 - **Idle connections and timeouts (G-026, both layouts):** the `POOL_MAX_*` values are caps. Idle,
   every framework pool shrinks to `POOL_MIN_IDLE` (default 1) after `POOL_IDLE_TIMEOUT_S` (default
   600), so an idle process holds about 4 connections. Optional `APP_DB_QUERY_TIMEOUT_MS`
@@ -830,7 +891,8 @@ upload(@Req() req: Record<string, unknown>) {
   `AGENT_RUNTIME_URL` (`https:`, or `http:` in development/test or to a loopback host),
   `AGENT_RUNTIME_KEY_ID`, `AGENT_RUNTIME_KEY` (`secret-ref:` outside development/test).
 - **Who may use it.** Keys `identity:agent:chat` (use it), `identity:agent:restricted_data` (send
-  `restricted` data to an approved provider), `identity:agent:propose` (M3). A tenant also needs
+  `restricted` data to an approved provider), `identity:agent:propose` (M3: `propose` capabilities,
+  see Proposals below). A tenant also needs
   mode `synthetic` (org bootstrap `agent: { mode }`); each company is set on its own. `identity:*`
   holders get all three keys, including `restricted_data` (GAPS #23): grant `identity:agent:chat`
   explicitly on production roles.
@@ -877,8 +939,59 @@ upload(@Req() req: Record<string, unknown>) {
     masking (not built yet) is a prerequisite before real data.
   - Totals the assistant states come from `profile_sheet` or `aggregate`, never from adding rows it read.
     Ids inside file content reach the model as `[id]`.
-  - Not yet: the attach control in the scaffold UI (the client service supports `attachments`), the
-    workbook map (D2b), marking an upload `restricted`.
+  - **Workbook structure (G-037 G1).** Three more framework tools for xlsx and csv, for sheets that are
+    not "row 1 is the header" tables (matrix, transposed, stacked blocks, merged headers):
+    `files.list_sheets@1` (each sheet's state, computed used range, hidden rows and columns, counts of
+    merges, tables, charts, names, validations and formulas; a sheet over the 1,000,000-cell scan cap is
+    listed with `scanned: false`), `files.read_grid@1` (raw cells with A1 addresses for a range the model
+    picks, at most 400 cells and 16 KiB a call; `truncated.nextRange` gives the rest; a formula cell has
+    `f`: `x` external, `v` volatile, `f` other, and `v` is its cached value, never evaluated) and
+    `files.read_structure@1` (merges, formula groups as R1C1 shapes with string literals replaced by `"…"`,
+    tables, defined names, list validations, chart series refs; `parts`, `range`, `offset`). They return
+    structure only: deciding what a block means is your code's job (a consumer step, not a framework one).
+    An analyser older than the app is refused at boot (`analyser_version_skew`): upgrade the analyser image
+    together with `@astram/agent`. App code gets the same three reads, unmasked, on `FileAnalyser`
+    (`outlineWorkbook`, `readGrid`, `readStructure`).
+  - **Masking of these results, and its limits.** Like every `files.*` result they are masked before the
+    model sees them. `read_grid` has no header row, so the analyser marks each cell covered by a header hint
+    (`p: <kind>`, from the column or transposed row of a header such as `Name` or `Email`, even when the
+    header lies outside the range read) and the cell is masked whole. Emails, phones and the other built-in
+    kinds are masked wherever they appear, including table column names, chart titles, defined-name text
+    and validation lists. **Person names have no detector.** A name that no header hint covers reaches the
+    model unmasked: a table column that is itself a name, a chart title or series name, the text of a
+    defined name, a validation list outside a hinted column. Put the data the model must not see behind a
+    hinted header, or keep such files out of a conversation until your own detector (`defineDetector`)
+    covers them.
+  - **Disclosed over-mask.** The hint rule can mask more than the personal data: a leftmost header cell
+    that is itself a hint (`Name`) masks the other header labels of that row, and in a stacked sheet a
+    later block's `Name` label and the values under an earlier block's hinted column are masked too. This
+    fails safe (values are never shown); the model may need to ask for the labels by position.
+  - Not yet: the attach control in the scaffold UI (the client service supports `attachments`), marking
+    an upload `restricted`. The earlier "workbook map" design (D2b) was replaced by the three tools above.
+- **Proposals (G-036 M3, ADR-0049 M3 addendum).** The assistant never writes records; it may
+  **propose**, and a person applies.
+  - Declare what a change means with `defineProposalKind({ id, version, label, payloadSchema,
+    targetTypes: [oneAppType], keys: { propose, review, apply }, plan, validate?, preview?,
+    distinctApprover?, maxItems?, expiryMs? })` (`@astram/identity`) and pass it to
+    `IdentityFeatureModule.forRoot({ proposalKinds })`. `plan` turns a payload into `create` /
+    `update` items on that one type; `plan` and `validate` get a read-only `ctx.records`. There is no
+    apply handler: the framework writes the approved items through the record pipe as the approver,
+    in one transaction, with read-back. The three keys are custom actions on an app type (ADR-0038).
+  - A capability with `effect: 'propose'`, `proposes: [kindIds]` and `requires` including
+    `identity:agent:propose` returns a draft `{ kind, payload, title, rationale?, metadata? }`; the
+    framework submits it as the user. The model gets a handle and counts; the browser gets
+    `approval.required` (`AgentTaskState.approvals` in `@astram/frontend-core`). Workers cannot propose.
+  - To let a capability see proposals, declare `readsProposalKinds: [kindIds]` with
+    `identity:proposal:read` in `requires` and call `ctx.proposals.list({ kind, state?, page?,
+    pageSize? })` (≤ 20 a page, the same scope as `GET /identity/proposals`, handles and labels only).
+  - **Never read or write the `proposal` / `proposal_item` rows yourself**: the engine refuses every
+    `get`, `list`, `aggregate`, lock and write on them outside the framework's own lifecycle operation,
+    whatever keys the user holds. Use `/identity/proposals` (review UI: `createProposalService` in
+    `@astram/frontend-core`; render values as text and never display ids) or the capability reader.
+  - By default the requester cannot approve or apply their own proposal; `distinctApprover: false`
+    opts out and adds a caution to the kind's `review` / `apply` permission text. Terminal proposals
+    are deleted after `PROPOSAL_RETENTION_DAYS` (365, 30–3650); their audit rows stay.
+  - Not yet: a review page in the scaffolded template.
 
 ## Available, but NOT wired to a data path (do not assume they protect data yet)
 - **Envelope encryption / per-tenant DEKs** (`createEncryptionService`, `LocalKekCrypto`): the

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@astram/guide",
-  "version": "0.2.25",
+  "version": "0.2.26",
   "license": "MIT",
   "description": "Astram framework — the AI/dev GUIDE layer (ADR-0025). Ships the framework's hard rules + capabilities manifest as installed, upgrade-refreshed docs. A consumer app's CLAUDE.md references FRAMEWORK_RULES.md from this package; `npm update` replaces it wholesale so new rules arrive cleanly, while the app's own .astram/guide/ layer is never touched. This is the framework-owned FLOOR: an app rule may tighten it, never weaken it.",
   "type": "commonjs",