@ftisindia/create-app 0.1.5 → 0.2.0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ftisindia/create-app",
-  "version": "0.1.5",
+  "version": "0.2.0",
   "description": "One-command scaffolder for the Phase 1 NestJS foundation starter.",
   "type": "module",
   "bin": {

package/template/.env.example

@@ -13,6 +13,10 @@ AUTH_GOOGLE_ENABLED=false
 AUTH_GOOGLE_CLIENT_ID=
 AUTH_GOOGLE_CLIENT_SECRET=
 AUTH_GOOGLE_CALLBACK_URL=http://localhost:3000/auth/google/callback
+# The backend redirects the browser here with a one-time ?code= after Google
+# login. When a frontend app consumes this API (for example one scaffolded with
+# @ftisindia/create-frontend), point this at the frontend's login route, e.g.
+# AUTH_GOOGLE_SUCCESS_REDIRECT_URL=http://localhost:4200/login
 AUTH_GOOGLE_SUCCESS_REDIRECT_URL=http://localhost:3000/auth/google/success
 AUTH_GOOGLE_ERROR_REDIRECT_URL=http://localhost:3000/auth/google/error
 # Required when AUTH_GOOGLE_ENABLED=true. Use at least 32 random characters.
@@ -40,3 +44,27 @@ TEST_USER_PASSWORD=
 TEST_USER_DISPLAY_NAME=Starter Owner
 TEST_ORG_NAME=Demo Organisation
 TEST_ORG_SLUG=demo
+
+# Form builder (@ftisindia/form-builder)
+FORMS_OUTBOX_ENABLED=true
+FORMS_OUTBOX_POLL_MS=5000
+FORMS_OUTBOX_HEARTBEAT_MS=60000
+FORMS_FILE_GC_INTERVAL_MS=3600000
+FORMS_FILE_TEMP_TTL_HOURS=24
+FORMS_MAX_UPLOAD_MB=25
+FORMS_FILE_STORAGE_DIR=./var/uploads
+# Set to off only to bypass the boot-time engine schema check (not recommended).
+FORMS_SCHEMA_CHECK=on
+
+# Report builder (@ftisindia/report-builder)
+# Boot-time schema check for the engine tables (set off only to bypass temporarily).
+REPORTS_SCHEMA_CHECK=on
+# Optional HMAC secret for report cursors/bulk-action tokens (min 32 chars).
+# When empty, a dedicated key is derived from JWT_SECRET via HKDF.
+REPORTS_TOKEN_SECRET=
+# Reports-owned async export worker + file storage (no forms dependency).
+REPORTS_EXPORT_WORKER_ENABLED=true
+REPORTS_EXPORT_POLL_MS=5000
+REPORTS_EXPORT_STORAGE_DIR=./var/report-exports
+REPORTS_EXPORT_RETENTION_DAYS=7
+REPORTS_EXPORT_RETENTION_SWEEP_MS=3600000

package/template/README.md

@@ -117,6 +117,28 @@ store the selected `orgId`, then call
 `permissionKeys` for route, menu, and button visibility in the frontend. Backend
 guards remain the source of truth for access control.
 
+A companion frontend starter exists: `npx @ftisindia/create-frontend my-app`
+scaffolds an Angular app already wired to these endpoints.
+
+### Google OAuth with a frontend
+
+`GET /auth/google/callback` redirects the browser to
+`AUTH_GOOGLE_SUCCESS_REDIRECT_URL` with a one-time `?code=` that the frontend
+must exchange via `POST /auth/oauth/exchange`. The default value points at the
+backend's own origin, so set it to the frontend's login route (for example
+`http://localhost:4200/login`) or the code never reaches the frontend.
+
+### Exporting the OpenAPI document
+
+Frontend tooling can consume the API contract without a running server:
+
+```bash
+npm run export:openapi
+```
+
+This writes `docs-json.json` (the same document served at `/docs-json`) to the
+project root. No database connection is required.
+
 ## Architecture Rules
 
 - JWTs contain identity only. Organisation membership, roles, and permissions are read per request.
@@ -126,6 +148,35 @@ guards remain the source of truth for access control.
 - Mutating business flows should write audit rows.
 - Setup and CI commands are non-destructive. Use `npm run db:reset` only when you explicitly want to reset a local database.
 
+## Dynamic Forms
+
+The app ships with a definition-driven form engine (`@ftisindia/form-builder`
+plus the `src/modules/forms` glue module). Form definitions are versioned JSON
+documents created over the API; submissions run a validated, transactional
+action pipeline with a reliable outbox for emails/webhooks. Most new forms
+need zero backend code — register a field type, action, or data source only
+when you need genuinely new behavior.
+
+- Guide: `docs/FORMS.md` (endpoints, extension story, webhooks, public forms, files, upgrades)
+- Completion standard: `docs/FORMS_CHECKLIST.md`
+- Example definitions: `src/modules/forms/examples/` (registration, abstract submission, login-as-a-form)
+- Definitions as code: `npm run gen:form -- <key>` scaffolds a definition + lint spec; `npm run forms:push` loads it into an org (linted, versioned, audited)
+
+## Dynamic Reports
+
+The app also ships with a definition-driven report engine
+(`@ftisindia/report-builder` plus the `src/modules/reports` glue module).
+Report definitions are versioned JSON views over registered data sources —
+form submissions are one convenient kind, any table/join is another — with
+filtering, sorting, search, keyset pagination, drift-safe bulk row actions,
+tags, saved views, and snapshot-consistent CSV/XLSX exports through one
+declarative API. Performance is enforced at publish time: unindexed shapes
+fail to publish and the error carries the exact migration SQL to fix them.
+
+- Guide: `docs/REPORTS.md` (QuerySpec, sources, tiers, actions, exports, upgrades)
+- Completion standard: `docs/REPORTS_CHECKLIST.md`
+- Example definitions: `src/modules/reports/examples/` (org-members over a custom source, abstract-review-board over a form)
+
 ## Adding Features
 
 Use the sample module as the copy-paste reference:

package/template/_gitignore

@@ -5,3 +5,9 @@ coverage/
 .env
 .env.*
 !.env.example
+
+# Generated by npm run export:openapi
+docs-json.json
+
+# Form-builder local file storage
+var/

package/template/_package.json

@@ -6,8 +6,11 @@
   "scripts": {
     "build": "nest build",
     "db:create": "node scripts/db-create.mjs",
+    "export:openapi": "ts-node scripts/export-openapi.ts",
     "format": "prettier --write .",
     "format:check": "prettier --check .",
+    "forms:push": "ts-node scripts/push-form.ts",
+    "gen:form": "node scripts/gen-form.mjs",
     "gen:module": "node scripts/gen-module.mjs",
     "lint": "eslint .",
     "prepare": "husky || true",
@@ -24,12 +27,14 @@
     "setup:ci": "node scripts/setup-local.mjs --yes",
     "test": "jest",
     "test:cov": "jest --coverage",
-    "test:e2e": "node scripts/test-db.mjs && jest --config test/jest-e2e.json --runInBand",
+    "test:e2e": "node scripts/test-db.mjs && node ./node_modules/jest/bin/jest.js --config test/jest-e2e.json --runInBand",
     "test:watch": "jest --watch",
     "db:reset": "prisma migrate reset"
   },
   "dependencies": {
     "@casl/ability": "6.8.1",
+    "@ftisindia/form-builder": "^0.1.0",
+    "@ftisindia/report-builder": "^0.1.0",
     "@nestjs/common": "11.1.24",
     "@nestjs/config": "4.0.4",
     "@nestjs/core": "11.1.24",
@@ -56,6 +61,7 @@
     "@nestjs/schematics": "11.1.0",
     "@nestjs/testing": "11.1.24",
     "@types/jest": "30.0.0",
+    "@types/multer": "1.4.13",
     "@types/node": "20.19.41",
     "@types/passport": "1.0.17",
     "@types/passport-google-oauth20": "2.0.17",
@@ -78,6 +84,9 @@
     "typescript-eslint": "8.60.0",
     "typescript": "5.9.3"
   },
+  "overrides": {
+    "js-yaml": "4.2.0"
+  },
   "lint-staged": {
     "*.{js,mjs,ts,json,md,yml,yaml}": "prettier --write",
     "*.{js,mjs,ts}": "eslint --fix"

package/template/docs/FORMS.md

@@ -0,0 +1,188 @@
+# Forms Module (`@ftisindia/form-builder`)
+
+This app ships with a definition-driven dynamic form engine. Form definitions
+are versioned JSON documents in the database; behavior — field types, button
+actions, data sources — is code this app registers. New forms are **data**:
+most ship with zero backend code.
+
+- Engine architecture: the `form-builder-design.md` / ecosystem guide pair in
+  the starter repository.
+- Completion standard: [`FORMS_CHECKLIST.md`](./FORMS_CHECKLIST.md) on top of
+  [`MODULE_COMPLETION_CHECKLIST.md`](./MODULE_COMPLETION_CHECKLIST.md).
+
+## The 60-second tour
+
+```text
+POST   /organisations/:orgId/forms                                  forms.create   (new draft version)
+PATCH  /organisations/:orgId/forms/:key/versions/:v                 forms.update   (drafts only)
+POST   /organisations/:orgId/forms/:key/versions/:v/publish         forms.publish  (lints, freezes, archives prior)
+POST   /organisations/:orgId/forms/:key/versions/:v/public-access   forms.managePublicAccess (audited)
+GET    /organisations/:orgId/forms/:key/render                      formSubmissions.create (definition + options)
+POST   /organisations/:orgId/forms/:key/submissions                 formSubmissions.create { mode, data, ... }
+POST   /organisations/:orgId/forms/:key/files?field=<name>          formSubmissions.create (upload before submit)
+GET    /organisations/:orgId/forms/:key/submissions/export          formSubmissions.export (audited PII egress)
+GET    /public/organisations/:orgId/forms/:key/render|submissions   (anonymous, public forms only)
+```
+
+Example definitions live in `src/modules/forms/examples/` — registration
+(plain persist), abstract-submission (nested repeatable groups, data source,
+rules, file upload), and login (public form whose submit action delegates to
+the real `AuthService`).
+
+## How it executes
+
+A submit runs: coerce → overwrite `source: 'context'` hidden fields from the
+request context (client values for them are always discarded) → lifecycle
+hooks → full validation (JSON Schema in draft or submit mode + bounded
+json-logic rules + data-source membership + file references) → the action
+pipeline: validation actions, then **all transactional actions in ONE
+database transaction** (submission, action logs, audit rows, outbox rows
+commit or roll back together), then post-commit side effects via the
+**transactional outbox** (in-process poller with Postgres row-lock claiming,
+idempotent enqueue, claim-token fencing, active-job heartbeat, retries with
+backoff, parks as FAILED after max attempts; delivery writes an audit row). A
+failed email never rolls back a valid submission; a failed transactional step
+rolls back everything.
+
+## Adding behavior (the three registries)
+
+Implement the engine interface, decorate the provider, list it in a module —
+the registry bootstrap finds it at startup:
+
+```ts
+import { FormDataSource } from './modules/forms/infrastructure/registry/form-extension.decorators';
+import type { DataSourceDef, DataSourceContext, DataSourceOption } from '@ftisindia/form-builder';
+
+@Injectable()
+@FormDataSource()
+export class TrackSource implements DataSourceDef {
+  key = 'conference-tracks';
+  constructor(private readonly prisma: PrismaService) {}
+  async fetch(
+    _params: Record<string, unknown>,
+    ctx: DataSourceContext,
+  ): Promise<DataSourceOption[]> {
+    return this.prisma.track
+      .findMany({
+        where: { orgId: ctx.orgId, active: true },
+        select: { id: true, name: true },
+      })
+      .then((rows) => rows.map((row) => ({ value: row.id, label: row.name })));
+  }
+}
+```
+
+Same pattern with `@FormFieldType()` (implement `FieldTypeDef`),
+`@FormActionHandler()` (implement `FormAction`), `@FormLifecycleHook()`
+(implement `FormLifecycle`). Builtin actions: `validateAll`, `persist`,
+`persistDraft`, `lockEditing`; shipped app actions: `sendConfirmationEmail`
+(post-commit, enqueues email), `authenticate` (dangerous, delegates to
+`AuthService.login`).
+
+## Dangerous actions
+
+An action with `dangerous: true` (payments, auth, anything with teeth) is
+triple-gated: wiring it into a definition requires `forms.wireDangerous` AND
+membership in the org's `forms.allowedDangerousActions` typed setting (both
+checked at save/publish); at execute time the allowlist is re-checked. The
+linter also refuses pipelines combining `authenticate` with `persist` —
+credentials are never stored, and dangerous actions' input/output are logged
+as `[REDACTED]`.
+
+## Public forms
+
+Flip a published definition with `POST .../public-access { "access": "public" }`
+(requires `forms.managePublicAccess`, writes an audit row). Public routes are
+throttled tighter than the app default; the engine enforces the per-IP/day
+cap (`forms.maxSubmissionsPerIpPerDay` setting, per-form override in the
+definition's `settings`) and the captcha seam (`settings.captcha: true` —
+publish fails closed unless a `CaptchaVerifier` is bound to the
+`FORMS_CAPTCHA_VERIFIER` token). Public forms cannot contain file fields.
+
+## Files
+
+Upload before submit (`POST .../files?field=<name>`); the submission carries
+only `{ "fieldName": { "fileId": "..." } }`. The server sniffs the MIME type
+from content, enforces field + org size caps, computes a sha256, and binds
+the file to uploader AND org. Submit-time linking re-verifies ownership
+in-transaction and promotes TEMPORARY → LINKED; orphans are garbage-collected
+after `FORMS_FILE_TEMP_TTL_HOURS`. Bytes live under `FORMS_FILE_STORAGE_DIR`
+(swap `LocalDiskStorageAdapter` for S3/GCS without engine changes).
+
+## Webhooks
+
+A definition can declare webhook destinations that fire when a submission is
+**submitted** (never for drafts, never for pipelines without `persist`):
+
+```jsonc
+"settings": {
+  "webhooks": [
+    { "url": "https://hooks.example.com/forms", "secret": "rotate-me", "includeData": true }
+  ]
+}
+```
+
+- **Reliable by construction:** the job is written to the transactional outbox
+  inside the same transaction as the submission, then delivered post-commit
+  with retries/backoff (parks as FAILED after max attempts; success writes a
+  `forms.outbox.delivered` audit row). The Prisma store claims jobs with
+  `FOR UPDATE SKIP LOCKED`, assigns an opaque claim token, heartbeats active
+  `PROCESSING` rows with that token, and guards `DONE`/`PENDING`/`FAILED`
+  transitions with the same token so stale workers cannot settle reclaimed
+  jobs. A duplicate idempotency key is a no-op so retries do not roll back the
+  submission. Idempotency key: `<submissionId>:webhook:<index>`, also sent as
+  `x-forms-idempotency-key` for receiver-side dedupe.
+- **SSRF-gated:** the destination host must be in the org's audited
+  `forms.webhookAllowedHosts` typed setting (default empty = webhooks
+  disabled), and the URL must be `https` — `http` is allowed for loopback
+  hosts only (local development). Enforced at save AND publish lint
+  (`WEBHOOK_HOST_NOT_ALLOWED`, `WEBHOOK_URL_INSECURE`, …). Delivery does a
+  fresh DNS resolution, rejects private/reserved/link-local destinations in
+  `NODE_ENV=production`, pins the request to the resolved IP while preserving
+  Host/SNI, and refuses redirects instead of following them.
+- **Payload:** `{ event: 'submitted', orgId, formKey, formVersion, submissionId,
+occurredAt, data? }` — `data` passes sensitive-field redaction; set
+  `includeData: false` to send a pure notification.
+- **Signed:** with a `secret`, deliveries carry
+  `x-forms-signature: sha256=<hmac-sha256(rawBody)>`. The engine computes the
+  signature before enqueueing; the outbox payload stores the raw body and
+  signature, not the shared secret. Old-format jobs containing `secret` are
+  sanitized to the new `{ rawBody, signature }` payload before delivery. The
+  secret lives only in the definition
+  document (DB) — treat it as a rotatable shared token, not a master
+  credential. Max 5 webhooks per definition.
+
+## Definitions as code (`gen:form` + `forms:push`)
+
+```bash
+npm run gen:form -- customer-feedback            # scaffolds definition + lint spec
+npm test                                         # spec lints the JSON, no DB needed
+npm run forms:push -- --file src/modules/forms/definitions/customer-feedback.form.json \
+    --org demo --user owner@example.com --publish
+```
+
+`gen:form` writes `src/modules/forms/definitions/<key>.form.json` and
+`test/<key>.form.spec.ts` (meta-schema + publish-stage lint — fails `npm test`
+the moment the definition would be rejected by the API). `forms:push` loads a
+definition file into an org through the real engine path: it impersonates an
+existing member (their RBAC applies), so every push is linted, versioned, and
+audited exactly like an HTTP call. Re-pushing an unchanged file is a no-op;
+changed content creates the next version (add `--publish` to go live).
+
+## Org settings & env
+
+Typed settings: `forms.allowedDangerousActions` (string[]),
+`forms.maxFileSizeMb`, `forms.enableRuleIteration`, `forms.virusScanRequired`
+(seam only — no scanner shipped), `forms.maxSubmissionsPerIpPerDay`.
+Env knobs (see `.env.example`): `FORMS_OUTBOX_ENABLED`, `FORMS_OUTBOX_POLL_MS`,
+`FORMS_FILE_GC_INTERVAL_MS`, `FORMS_FILE_TEMP_TTL_HOURS`, `FORMS_MAX_UPLOAD_MB`,
+`FORMS_FILE_STORAGE_DIR`, `FORMS_SCHEMA_CHECK`.
+
+## Engine upgrades (schema ownership)
+
+This app owns `schema.prisma` and the migration history. When a new engine
+version changes its data model: read its changelog, copy the updated snippet
+from `node_modules/@ftisindia/form-builder/prisma/forms.prisma` into
+`prisma/schema.prisma`, run `npx prisma migrate dev`. The boot-time schema
+check fails fast with exactly that instruction if the database and engine
+ever disagree (`FORMS_SCHEMA_CHECK=off` is the emergency hatch).

package/template/docs/FORMS_CHECKLIST.md

@@ -0,0 +1,69 @@
+# Forms Module Completion Checklist
+
+The forms module must pass `MODULE_COMPLETION_CHECKLIST.md` verbatim, plus
+these form-builder-specific items from the ecosystem guide. Re-verify after
+any engine upgrade or forms-module change.
+
+Last verified on 2026-06-13 against the generated dogfood app (`test-out`) with
+local PostgreSQL `postgres/root`: `npm run dogfood`, generated app
+`npm run build`, `npm run lint`, `npm test`, `npm run test:e2e`; engine
+`npm test`, `npm run lint`, `npm run typecheck`.
+
+## Boundary
+
+- [x] The engine package (`@ftisindia/form-builder`) compiles and unit-tests without the template; only `src/modules/forms` imports template services.
+- [x] The engine's boundary lint passes: no `@nestjs/*`, `@prisma/*`, `@casl/*`, or template imports anywhere in the engine core; engine errors are engine-typed and mapped to HTTP exceptions only in the glue (`forms-error.mapper.ts`).
+
+## Schema ownership
+
+- [x] Boot-time schema check proven both ways: green on a freshly migrated app; clear fail-fast message naming the engine schema version and snippet path on a deliberately stale schema.
+- [x] `prisma/forms.prisma` in the engine matches the models in the app's `schema.prisma`; upgrades follow copy-snippet -> `prisma migrate dev`.
+
+## Permissions & registry
+
+- [x] All 11 form permission keys exist in `permissionKeys`, are seeded, and `test/forms-permission-sync.spec.ts` passes (engine <-> template set equality).
+- [x] Every protected form route is in `routePermissionRegistry` (the boot-time registry validator passes).
+
+## Gating
+
+- [x] Attach-time gating proven by test: lacking `forms.wireDangerous` cannot save/publish a definition wiring a `dangerous` action; the `forms.allowedDangerousActions` org setting is respected at save/publish AND at execute time.
+- [x] `forms.managePublicAccess` gating proven: lacking it cannot set or unset `settings.access = 'public'` (including publishing an already-public draft); every flip writes an audit row with previous/next.
+
+## Tenancy
+
+- [x] Cross-org tests: definition fetch, submission read, and file reference across orgs are rejected as not-found.
+
+## Transactionality & outbox
+
+- [x] Phase-2 transactionality proven: an induced `lockEditing` failure rolls back persist and in-tx action logs and enqueued outbox rows.
+- [x] Outbox worker runs inside `requestContext.run({ source: 'worker', jobId, orgId, userId, requestId })`; retries honor backoff; exhausted jobs park as `FAILED`; an audit row (`forms.outbox.delivered`) is written on final success.
+- [x] Duplicate outbox enqueue with the same idempotency key is a no-op, not a submission-rolling unique violation.
+- [x] Claimed jobs carry an opaque `claimedBy` lease token; heartbeat and `DONE`/retry/failed terminal transitions are guarded by that token so stale workers cannot settle reclaimed jobs.
+- [x] Active `PROCESSING` jobs heartbeat while handlers run; e2e covers both the store primitive and the actual dispatcher wiring with a slow handler.
+
+## Public forms
+
+- [x] Public path: `@Public()` render/submit works without membership; a non-public form 404s on the public routes; org-scoped storage still enforced; permissioned paths unaffected.
+- [x] Tier-1 abuse baseline active: public routes throttled tighter than the app default; per-IP/day caps read from typed settings (with per-form override) and enforced; IP/UA stamped on anonymous submissions; captcha seam invoked when enabled; captcha-enabled-without-adapter fails at publish, not at submit.
+
+## Files
+
+- [x] Upload -> submit -> `LINKED` flow proven; a foreign user's `fileId` is rejected (IDOR); GC sweeps orphaned `TEMPORARY` files past TTL; public forms with file fields are rejected at publish.
+
+## Webhooks
+
+- [x] Webhook destinations are gated by the `forms.webhookAllowedHosts` setting at save AND publish (`WEBHOOK_HOST_NOT_ALLOWED`); https required except loopback (`WEBHOOK_URL_INSECURE`).
+- [x] Webhook jobs enqueue inside the submit transaction (idempotency key `<submissionId>:webhook:<i>`), never on draft saves or persist-less pipelines.
+- [x] Deliveries are signed (`x-forms-signature`) when a secret is set and carry `x-forms-idempotency-key` for receiver-side dedupe; new outbox payloads store the precomputed signature, not the shared secret; legacy secret-bearing payloads are sanitized before delivery; payload data passes sensitive-field redaction; failures retry then park as FAILED.
+- [x] `forms:push` loads definitions only through the audited service path (real member RBAC, save-time lint, versioning) and never raw DB writes.
+
+## Export & redaction
+
+- [x] `formSubmissions.export` enforced independently of `.read`; every export writes an audit row with form key, filters, row count, and included fields.
+- [x] Credential/sensitive fields never appear in `FormActionLog` or `AuditLog` metadata (redaction test); dangerous actions log `[REDACTED]` for input and output; unknown credential-like keys (`apiKey`, `token`, etc.) are defensively redacted.
+
+## Envelope & docs
+
+- [x] Engine validation errors arrive as `{ error: { code, message, details: { errors: [...] } } }` through the global filter; lint failures carry `details.issues`.
+- [x] Engine-thrown denials and not-founds are indistinguishable in shape from the template's own.
+- [x] Swagger documents all form routes (dynamic payloads as `object`).