@astram/guide 0.2.4 → 0.2.6

package/CAPABILITIES.md

@@ -44,8 +44,19 @@ imports: [
   `identityControlPlaneKeys?`, `config?`, and optional overrides `credentialResolver?`,
   `profileResolver?`, `credentialWriter?`, `resetTokenStore?`, `recoveryCodeStore?`,
   `profileWriter?`, `sessionSweeper?`, `auditSink?`, `mfaDeviceStore?`, `totpVerifier?`,
-  `sessionFactsProjector?`, `scopeSubtreeResolver?`, `membershipChecker?`. With `auth: true` every
-  override has a MariaDB/Redis default; leave them out unless you have a reason.
+  `sessionFactsProjector?`, `scopeSubtreeResolver?`, `membershipChecker?`, and
+  `storage?: ObjectStore | 'env'` (file storage, below). With `auth: true` every override has a
+  MariaDB/Redis default; leave them out unless you have a reason.
+- **Secrets at boot** — `main.ts` calls `await resolveBootSecrets()` (`@astram/guard`) first,
+  then loads `AppModule` with `await import('./app.module')` (the scaffold already does). Every env
+  value written as `secret-ref:<path>` is replaced with the secret before any `forRoot` reads env.
+  Provider: `SECRETS_PROVIDER` = `none` (default) | `encrypted-file` (`SECRETS_FILE`,
+  `SECRETS_KEK_FILE` (a mounted 32-byte key file, never an env value), `SECRETS_DEK_FILE`); `vault`
+  is reserved and refused. A missing, unreadable or corrupt `SECRETS_FILE` aborts boot naming the
+  variable, even before any reference uses it. An unresolvable reference, or a reference with no provider, aborts
+  boot (env key and path only, never a value). If `main.ts` skips the call while a reference is in
+  env, `ShieldModule` refuses to boot (`assertNoUnresolvedSecretRefs`). Do not import `AppModule`
+  statically in `main.ts`.
 - **`IdentityFeatureModule.forRoot({ rootIds?, extraObjects? })`** (`@astram/identity`). Pass the
   SAME `rootIds` as `ShieldModule`. `extraObjects` (your own registered object types) are given
   HERE ONLY: the module builds the record authorization from them and `ShieldModule` decides with
@@ -123,8 +134,8 @@ imports: [
   (099/100) run only with `NODE_ENV=development` or `ASTRAM_SEED_DEV=1`.
 
 ## The record pipe (`@astram/engine`)
-- **`RecordEngine`** (token `RECORD_ENGINE`) / `RecordsService` — create/get/update/delete/list
-  over registered object types through one pipe: permission gate (`decide()`) → server-trusted
+- **`RecordEngine`** (token `RECORD_ENGINE`) / `RecordsService` — create/get/update/delete/list,
+  plus `aggregate` and `createMany`, over registered object types through one pipe: permission gate (`decide()`) → server-trusted
   write context → validate → write → junction sync → audit (mandatory) → event.
 - **Record decision inputs**: `get`, `update` and `delete` read the stored row first and pass its
   fields and tenant to `decide()`, so record rules such as "deny update where status eq locked"
@@ -134,6 +145,44 @@ imports: [
   (record rules, owner shares, cross-tenant grants, subtree), filtered in SQL; `get()` uses the
   same scope. Membership/subtree-scoped lists ("companies I belong to") are provided.
 - Stable paging: every `list()` orders by the primary key last.
+- **`list()` filters** (`where`, field key → value, ANDed): a scalar is equality (`null` ⇒
+  `IS NULL`); or exactly one operator object per field: `eq`, `in` (≤ 1000 values; `[]` matches
+  nothing), `gt`/`gte`/`lt`/`lte`, `between: [lo, hi]` (range operators on `string`, `number`,
+  `uuid`, `timestamp` only), `isNull: true`, `isNotNull: true`. Every operand is validated and
+  coerced like a write (bad operand ⇒ `ValidationError`; uuids lower-cased, booleans bound as
+  1/0, timestamps as `Date`) and bound as a parameter. An unknown field or operator, two
+  operators in one object, or an operator not allowed for the field type (`eq`/`in` on `json`,
+  ranges on `boolean`/`json`) throws `RegistrationError`; nothing degrades to "no filter".
+  Only a scalar `null` means `IS NULL` (or use `isNull: true`): `{ eq: null }` and a `null`
+  inside `in` are rejected with `ValidationError`. A reversed `between` (`lo > hi`) is not
+  swapped and matches nothing. A `number` operand of ±Infinity is a `ValidationError`.
+  Filters run inside the same scoped query, so `list ≡ decide()` still holds.
+- **Behaviour change (engine 0.7):** equality in `where` is now validated and coerced like a
+  write; `eq` on json fields is rejected; scalar `null` now means `IS NULL` (it matched nothing
+  before). A key present with the value `undefined` (`where: { x: undefined }`) now throws
+  `ValidationError`: when spreading optional filters into `where`, omit undefined keys. An invalid timestamp (an invalid `Date`, `NaN` or ±Infinity epoch) is now a
+  `ValidationError` on writes too, and so is ±Infinity on a `number` field (writes and filters).
+- **`aggregate(typeKey, { where?, search?, groupBy?, measures, limit? })`**: `count` / `sum` /
+  `avg` / `min` / `max` over exactly the rows `list()` returns for the same `where`/`search`.
+  Gated as object-type `read` like `list()`; writes no audit row. `groupBy` ≤ 3 non-json keys;
+  1–10 measures, each `{ fn, field?, as }` with `as` matching `^[a-z_][a-z0-9_]{0,63}$`;
+  `sum`/`avg` on `number` only; `min`/`max` on `string`, `number`, `uuid`, `timestamp` (returns
+  the column's value, a `Date` for timestamps). Groups are ordered by the group keys; ungrouped ⇒
+  one group, grouped with no rows ⇒ `groups: []`. `limit` caps the number of groups (default 100,
+  max 1000).
+  - **Silent truncation:** a grouped aggregate returns at most `limit` groups and gives no signal
+    that more exist. If the number of groups can exceed `limit`, count the groups first or narrow
+    `where`.
+  - **DECIMAL precision:** `sum`/`avg` (and numeric `min`/`max`) come back as JS `number`. An
+    integer outside the safe range throws `EngineUnavailableError`, but a DECIMAL result that
+    does not fit a double is returned rounded. Do not use `aggregate` for exact money totals on
+    DECIMAL columns.
+- **`createMany(typeKey, inputs)`**: 1–500 rows per call (otherwise `ValidationError` before any
+  write). Each row runs the full single `create()` pipe (gate, validate, create wall, audit) in
+  one transaction, in input order; ids come back in input order. The first failing row throws
+  and the whole batch rolls back, audit rows included. Each row gets its own audit event tagged
+  with the call's `batchId` and the row's `batchIndex` (ADR-0039). Inside `transaction(fn)` it
+  joins the outer transaction.
 - `transaction(fn)` runs several pipe calls in one unit of work (rolls back on any failed write);
   `lockForUpdate(typeKey, id)` serialises a check-then-write.
 - `@Field({ immutable: true })` create-only fields; `trusted` fields are stamped server-side.
@@ -149,12 +198,167 @@ imports: [
   (a Rule-4 STOP). Field types this version: `string`, `number`, `boolean`, `uuid`, `timestamp`,
   `json`.
 
+## Custom object actions (ADR-0038; engine 0.7, guard 0.14, identity 0.11)
+- **Declare domain verbs on the type**: `@ObjectType({ ..., actions: ['activate', 'review'] })`.
+  The type's vocabulary is `read, create, update, delete` plus the declared actions, and each one
+  is a Gated permission key `identity:<typeKey>:<action>` on the type's `access.plane`.
+  `crudActions: false` drops the CRUD verbs (for authority-only types never written through the
+  engine, which then refuses create/update/delete on them; engine get/list on such a type without
+  a `read` action are denied by the gate, not refused pre-SQL like the writes); it needs a
+  non-empty `actions`.
+- Action names: snake_case (`^[a-z][a-z0-9_]*$`), at most 64 characters, no duplicates; `*` and
+  `access` are reserved; a CRUD verb in `actions` is rejected unless `crudActions: false`. A bad
+  declaration fails boot with `RegistrationError` naming the type and action.
+- **Gate a route on it**: `@Classified({ typeKey: '<type>', action: 'activate' })`. List the type in
+  `IdentityFeatureModule.forRoot({ extraObjects })` (the decision authority and the catalog only
+  know those types). An action the type does not declare still aborts boot (`verifyGatable`).
+  Without the grant the route is 403.
+- **Record-level check in the handler**: `records.authorize(typeKey, id, 'activate')` (also on
+  `BoundRecords` and the `transaction(fn)` handle). It runs the type gate, reads the row by id in
+  the caller's tenant, and decides at record level (record rules and owner shares for that
+  action or `*`). Allow resolves `void`; deny, a missing row and another tenant's row all throw
+  the same `AuthorizationDeniedError` (403); an undeclared action throws `RegistrationError`
+  (500) before any query. It returns no row data, writes no audit row and is not a list filter.
+- **A handler that writes needs two grants**: `authorize()` for the custom action, then the
+  write through the normal pipe (`records.update(...)`), which needs `update` and is audited.
+  There is no write under the custom action's key. With `activate` alone the update is 403. To
+  keep check and write consistent, run both inside `records.transaction(fn)`.
+- **Two behaviours of `authorize()` to know**:
+  - On a type with `readScope`, a non-root caller is denied for a row outside its write scope
+    (the same wall `update` uses), even when the custom action reads like a non-write check.
+  - Inside `transaction(fn)` it reads the row `FOR UPDATE` (a row lock held until commit) even
+    though it writes nothing; outside a transaction it is a plain read.
+- **Admin surface picks them up**: the permission catalog (`GET /identity/permission-catalog`),
+  role grants, the grant ceiling, `identity:<type>:*` wildcards and `@AccessBundle` all accept
+  the declared actions. A record share or a record policy for an action the type does not
+  declare is a 400 (`*` still allowed). A record policy on action `create` is also a 400
+  (`InvalidPolicyError`): record rules are evaluated per record and never apply at create, so
+  gate creation with the type's `create` grant; a `*` rule applies to the other actions. The
+  scaffolded access form shows each type's extra
+  actions as extra checkboxes.
+- **Auth facts**: pass `buildIdentityAuthKeys(extraObjects)` (`@astram/identity`, returns
+  `{ actionKeys, controlPlaneKeys }`) to `ShieldModule.forRoot({ identityActionKeys,
+  identityControlPlaneKeys })` with the SAME `extraObjects` array, so `/identity/auth-facts`
+  includes your types' custom keys. The plain `identityAuthActionKeys` /
+  `identityControlPlaneKeys` cover identity's own types only.
+- **Changing the action list later**:
+  - Removing a declared action orphans its catalog row: new grants and bundles on it fail, held
+    keys stop counting in auth facts, and a route classified on it fails boot. Re-adding the action
+    reactivates the row, and existing role grants on it take effect again. If that is not wanted,
+    revoke the grants before removing the action.
+  - Record shares and record rules with action `*` cover custom actions added later. Review
+    wildcard shares and rules when you add a sensitive action.
+  - A share whose action has since been removed can still be revoked: `unshare` skips the
+    declared-action check (revoking cannot grant). An unknown type is still a 400.
+- `actionsOf(reg)` (`@astram/guard`) returns a registration's vocabulary. `actionsFor(typeKey)`
+  is deprecated and returns only the CRUD verbs for every type.
+
+## File upload and object storage (`@astram/guard` + `@astram/identity`)
+One file per request, kept in a private S3/MinIO bucket with SSE-S3, with a `stored_file` record
+on the record pipe. You write no storage, MIME or authorization code.
+
+```ts
+// app.module.ts: ShieldModule.forRoot({ ..., storage: 'env' })  + IdentityFeatureModule.forRoot(...)
+import { Classified, FileIntake, FILE_INTAKE_REQUEST_KEY, type FileIntakeResult } from '@astram/guard';
+import { FilesService } from '@astram/identity';
+
+@Post('imports')
+@Classified({ typeKey: 'stored_file', action: 'create' })      // or your own type/action
+@UseInterceptors(FileIntake({ field: 'file', profile: 'xlsx' }))
+upload(@Req() req: Record<string, unknown>) {
+  const intake = req[FILE_INTAKE_REQUEST_KEY] as FileIntakeResult;
+  return this.files.store({ ...intake.file, bytes: intake.bytes });
+  // 201 { uuid, sha256, sizeBytes, contentType, originalName }; never the storage key
+}
+```
+
+- **`FileIntake({ field, profile, maxBytes? })`** — runs after the global guard, so a denied
+  caller's body is never read. Exactly one file in the named field, no other fields. Checks size,
+  extension, declared type and magic bytes; the stored type comes from the profile, never from the
+  client; the name is sanitized for display only. Errors (HTTP status + fixed reason code, never
+  the name or bytes): 400 `missing_file`, `too_many_files` (also a file sent under another field
+  name: multer reports it as an unexpected file), `empty`, `bad_request` (malformed or truncated
+  multipart, raw control characters in part headers); 413 `too_large`; 415 `type_not_allowed`,
+  `content_mismatch`. Raw UTF-8 and RFC 5987 (`filename*=UTF-8''…`) file names decode to the
+  same name. Size: the route cap defaults to the profile's configured cap
+  (`OBJECT_STORAGE_MAX_BYTES` under `storage: 'env'`, else 10 MiB); `maxBytes` can only lower it.
+  Profiles: `xlsx` only (50 MiB hard ceiling for any profile, route or env value). Needs
+  `@nestjs/platform-express` (an optional peer; the scaffold has it).
+- **`FilesService`** (provided by `IdentityFeatureModule`) — `store(intake)`, `open(uuid)`,
+  `remove(uuid)`. No actor argument: tenant, owner and audit actor come from the session. `store`
+  puts the object at `t/<tenant>/<uuid>`, then creates the record (gate, tenant stamp, audit); a
+  failed or denied create deletes the object. `remove` is a hard delete of record and object.
+- **`GET /files/:uuid`** (framework route) — `stored_file:read` decided per record: the uploader
+  reads by owner baseline, anyone else needs a grant or a share; access is never inherited from
+  your own record that points at the file. Missing and foreign answer the same 403; non-uuid 400.
+  The bytes are re-hashed against the stored sha256. Headers: `Content-Disposition: attachment`
+  (RFC 5987 name), `X-Content-Type-Options: nosniff`, `Cache-Control: no-store`, and
+  `Content-Type` from the record when it is an allowlisted type, else `application/octet-stream`.
+- **Status codes** — storage not configured (no `storage` option or `OBJECT_STORAGE_BUCKET`
+  unset) ⇒ 501 from `FilesService` and the route; store down, stored object missing or sha256
+  mismatch ⇒ 503 with a fixed message.
+- **Audit** — the create is the pipe's `admin_action` row; each read, delete and refused access
+  writes a `file_access` event (no key, name or bytes). `file_access` is a new `AuditEventType`
+  member: add a case if you `switch` over the union.
+- **Env** (`loadObjectStorageConfig`): `OBJECT_STORAGE_BUCKET` (unset ⇒ storage off),
+  `OBJECT_STORAGE_ENDPOINT` (`https:`; `http:` only in development/test or to a loopback host),
+  `OBJECT_STORAGE_REGION` (default `us-east-1`), `OBJECT_STORAGE_ACCESS_KEY_ID`,
+  `OBJECT_STORAGE_SECRET_ACCESS_KEY`, `OBJECT_STORAGE_SSE` (`AES256` only),
+  `OBJECT_STORAGE_MAX_BYTES` (plain decimal bytes, no leading zeros; the per-file cap for the
+  `xlsx` profile and every `FileIntake` route). Outside development/test the two credentials must be
+  `secret-ref:` values resolved at boot; a literal is refused. With `storage: 'env'` the bucket is
+  pinged at boot (probe put/get/delete under `_probe/`); failure aborts boot.
+- **Your xlsx parser must cap entry count and total uncompressed size.** The framework checks only
+  the zip header and `[Content_Types].xml`, not every entry, so a zip bomb in another part passes
+  the upload check.
+- **Lint wall** — the scaffold bans `multer`, `@aws-sdk/*`, `minio`, `fs` / `node:fs` (and
+  `/promises`) in `src/features/**`: use `FileIntake` and `FilesService`, never the bucket.
+- Do not grant `stored_file:create` to app roles; uploads go through `FilesService`.
+- Not provided: malware scanning, per-tenant encryption of file bytes (SSE-S3 only), streaming
+  downloads, dedup (each upload is its own record; `sha256` is returned so you can detect repeats).
+
+## Database names and several environments on one server (`SHIELD_DB_NAMESPACE`, ADR-0037)
+- The framework uses four databases, `<ns>_app`, `<ns>_identity`, `<ns>_authz` and `<ns>_audit`,
+  with its own accounts (`<ns>_core`, `<ns>_migrate`, `<ns>_audit_ops`, …). If you leave
+  `SHIELD_DB_NAMESPACE` unset, `<ns>` is `shield` (`shield_app`, …), which is unchanged from
+  earlier releases.
+- **To run dev and beta on one MariaDB server**, give each environment its own namespace. It must
+  be lowercase letters and digits only (`beta`, `staging2`; no `_`). Then:
+  1. Provision it with the framework's `db-provision --namespace beta` (from the astram-framework
+     repo, runbook §6). Use distinct passwords per namespace.
+  2. Set `SHIELD_DB_NAMESPACE=beta` for both the server and `npm run migration:run`. The same
+     value must reach both.
+  3. Run migrations (the ledger is `beta_app.schema_migrations`), then start the server.
+- Defaults follow the namespace: `APP_DB_USER` ⇒ `<ns>_core`, `APP_DB_NAME` ⇒ `<ns>_identity`,
+  `DDL_DB_USER` ⇒ `<ns>_migrate`. Explicit values win.
+- **Rules your app must follow:**
+  - `APP_DB_NAME` must be one of the four framework databases, even with the namespace unset.
+    Any other value refuses to boot.
+  - An invalid namespace refuses to boot and to migrate.
+  - Use a separate Redis (instance or DB index) per environment: Redis keys are not namespaced.
+  - Your own object types keep a literal `schema` or none. Do not hard-code `shield_*` names in
+    app SQL or migrations: read them from the injected `FRAMEWORK_SCHEMAS` (`@astram/guard`).
+- **Boot check:** outside development/test (an unset `NODE_ENV` counts as production), the server
+  refuses to start when framework migrations are missing from the ledger, no ledger is found, or
+  the ledger is unreachable. Run `migration:run` before starting. With the namespace unset, the
+  server looks for the ledger in `DDL_DB_NAME`, its own `APP_DB_NAME`, then `shield_app`, so
+  `migration:run` with the server's `.env` is found. If your migrate job uses a different
+  `APP_DB_NAME`, set the same `DDL_DB_NAME` for migrate and the server.
+- The `<ns>_backup` and `<ns>_monitor` accounts keep server-wide grants and can see every
+  namespace on the server.
+- One database per account (some managed hosts) is NOT supported yet (needs the four databases).
+
 ## Migrations (`@astram/migrate`)
 - `migrate` / `migrate:revert` / `migrate:status` with the deploy-time DDL account (`DDL_DB_*`);
   the running app never holds DDL. Framework migrations merge with yours (`APP_MIGRATION_DIR`).
 - Numeric ordering, duplicate-id detection, a database lock so concurrent deploys never apply a
   migration twice (`APP_MIGRATION_LOCK_TIMEOUT`, default 120 s), and drift detection (a changed
   applied file is reported; `--strict` fails on it). Never edit a migration that already ran.
+- Upgrade `@astram/migrate` together with `@astram/guard` and `@astram/identity`. The framework
+  migrations read the database names from the migrate context (`ctx.schemas`). An older migrate
+  works only while `SHIELD_DB_NAMESPACE` is unset, and stops with "upgrade @astram/migrate"
+  otherwise. On upgrade, `migrate:status --strict` reports the framework's rewritten migrations as
+  SUPERSEDED, not drift, and `migrate` updates their ledger rows without re-running them.
 
 ## Frontend (`@astram/frontend-core`, `@astram/sdk`, scaffolded `frontend/`)
 - `@astram/frontend-core`: auth, account, identity, audit, record-policy, share, cross-tenant and
@@ -170,9 +374,8 @@ imports: [
 
 ## Available, but NOT wired to a data path (do not assume they protect data yet)
 - **Envelope encryption / per-tenant DEKs** (`createEncryptionService`, `LocalKekCrypto`): the
-  engine exists, but no record field is encrypted and no production `DekStore` ships.
-- **Secrets provider** (`SecretsProvider`, `resolveSecretRefs`, `EncryptedFileSecretsProvider`):
-  the guard's runtime config does not resolve `secret-ref:` values at boot.
+  engine exists, but no record field is encrypted and no production `DekStore` ships. Stored
+  files are encrypted by the bucket (SSE-S3), not per tenant.
 - **Risk engine** (`createRiskEngine`): advisory scoring with no request-path caller.
 - **Hash-chained audit** (`createAuditRecorder`, `verifyChain`): the shipped audit writer is
   append-only but does not store the chain.
@@ -186,5 +389,6 @@ imports: [
 - External identity providers (OIDC, SAML, LDAP, Entra ID, Cognito, Okta, Google Workspace) and
   WebAuthn ceremonies.
 - Tenant- or role-mandated MFA (MFA is enforced for users who enrolled a device).
+- Single-database hosting (all framework tables in one database per account).
 
 If you need one of these, STOP and flag it in `.astram/guide/gaps.md` (Rule 4).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@astram/guide",
-  "version": "0.2.4",
+  "version": "0.2.6",
   "license": "MIT",
   "description": "Astram framework — the AI/dev GUIDE layer (ADR-0025). Ships the framework's hard rules + capabilities manifest as installed, upgrade-refreshed docs. A consumer app's CLAUDE.md references FRAMEWORK_RULES.md from this package; `npm update` replaces it wholesale so new rules arrive cleanly, while the app's own .astram/guide/ layer is never touched. This is the framework-owned FLOOR: an app rule may tighten it, never weaken it.",
   "type": "commonjs",