@zigrivers/scaffold 3.16.0 → 3.17.0

package/README.md

@@ -389,6 +389,7 @@ Every `scaffold init` wizard question can be answered via CLI flags, making scaf
 | `--backend-auth` | string | none, jwt, session, oauth, apikey |
 | `--backend-messaging` | string | none, queue, event-driven |
 | `--backend-deploy-target` | string | serverless, container, long-running |
+| `--backend-domain` | string | none, fintech |
 
 #### CLI Config Flags (require `--project-type cli` or auto-set it)
 
@@ -472,6 +473,33 @@ Every `scaffold init` wizard question can be answered via CLI flags, making scaf
 
 > **Flag aliases**: Game flags have `--game-*` aliases for consistency with other project types (e.g., `--game-engine` is equivalent to `--engine`). Bare flags like `--engine` still work.
 
+### Declarative init from a YAML manifest (`--from`)
+
+For multi-service projects, use `scaffold init --from <file>` to provide a
+full ScaffoldConfig as YAML instead of running the interactive wizard:
+
+```bash
+scaffold init --from services.yml --force
+```
+
+The file must be a complete ScaffoldConfig (with `version`, `methodology`,
+`platforms`, and `project.services[]`). Pass `-` to read from stdin.
+
+`--from` is exclusive with config-setting flags (`--methodology`, all
+`--backend-*`, `--web-*`, etc.); combining them is an error. Operational
+flags (`--root`, `--force`, `--auto`, `--verbose`, `--format`) still work.
+
+**Multi-service execution (v3.17.0+)**: Multi-service projects are
+fully executable. Every stateful command (`run`, `next`, `status`,
+`skip`, `complete`, `info`, `dashboard`, `decisions`, `reset`,
+`rework`) accepts `--service <name>` to target one service's pipeline.
+State is sharded under `.scaffold/services/<name>/state.json` with a
+merged global+service view; per-service locks are independent.
+Services can expose artifacts for cross-service consumption via the
+`exports` allowlist in config, and pipeline steps can declare
+`cross-reads:` in their frontmatter to consume foreign artifacts
+during assembly.
+
 #### How Flags Interact
 
 - **Flag > auto > interactive**: Flags always take highest precedence. `--auto --engine unreal` uses defaults for everything except engine.

package/content/knowledge/backend/backend-fintech-broker-integration.md

@@ -0,0 +1,244 @@
+---
+name: backend-fintech-broker-integration
+description: Multi-broker adapter pattern; credential rotation; error harmonization; rate-limit management; broker-side quirks.
+topics: [backend, fintech, brokers, integration, adapter-pattern, rate-limits, credentials, retry]
+---
+
+A fintech backend that routes orders or reads positions across more than one broker inherits the union of every broker's quirks, outages, auth schemes, and undocumented behaviors. The broker-integration layer exists to hide that mess behind one normalized internal contract so the rest of the system — risk, order lifecycle, ledger, UI — can stay clean. This doc covers the adapter contract, credential handling, error harmonization, rate-limit strategy, and the specific pitfalls that recur regardless of which brokers you connect.
+
+## Summary
+
+Brokers do not share conventions. Alpaca exposes REST+WebSocket with OAuth or static keys; Interactive Brokers' Client Portal uses a local gateway with session tokens that time out; Tradier is REST-only with bearer tokens; TD/Schwab uses OAuth with mandatory refresh flows; institutional venues speak FIX 4.2/4.4 over persistent TCP; some prime brokers ship vendor-specific binary protocols. Auth differs. Order-state taxonomies differ (Alpaca's `new/partially_filled/filled/canceled/expired` does not map 1:1 to IBKR's `Submitted/PreSubmitted/Cancelled/Filled`). Error-code schemes differ. Rate-limit headers differ — when they exist at all. Building "just one more broker-specific branch" into the order service is how a codebase becomes unmaintainable in 18 months. The adapter layer is the discipline that prevents that.
+
+The adapter contract is a narrow, normalized internal API: `placeOrder`, `cancelOrder`, `replaceOrder`, `fetchOrder`, `fetchFills`, `fetchPositions`, `fetchBalance`, `subscribeFills`, `subscribeQuotes`. Each broker-specific adapter implements that interface; everything above the interface — risk checks, order router, position service — speaks only the normalized types. Define the contract in an IDL (protobuf, OpenAPI, or just a well-typed TS/Go module) so schema drift is a compile error, not a runtime surprise. Normalized types include a canonical `OrderStatus` enum, a canonical `RejectReason` enum, and monetary fields in minor units (see `backend-fintech-ledger.md`).
+
+Credentials never live in source, never in plain env vars in production, and never in the container image. Store them in a secrets manager with audit trails (AWS Secrets Manager, GCP Secret Manager, HashiCorp Vault, 1Password Service Accounts). Fetch at process start, cache in memory with a TTL, refresh on expiry. Rotate on a schedule (30–90 days for static keys) and rotate immediately on any suspected compromise, employee offboarding, or vendor breach. Broker OAuth refresh tokens require a separate refresh worker — long-running processes with stale tokens are the single most common broker outage cause in practice.
+
+Error harmonization converts broker-specific failures into a small internal taxonomy the rest of the system can reason about: `transient` (retry with exponential backoff + jitter), `rate_limited` (backoff + queue, honor any `Retry-After`), `invalid` (non-retriable; surface to user with a clean reason), `auth` (refresh credentials and retry once; alert on repeat), `outage` (circuit-break, disable new-order flow, alert on-call), `unknown` (state is indeterminate — do NOT blindly retry; queue for reconciliation — see `backend-fintech-order-lifecycle.md`). The mapping table is the most audited piece of the adapter; mis-classifying `unknown` as `transient` causes duplicate orders and customer-visible incidents.
+
+Rate-limit management is client-side: a token-bucket per broker per credential (plus per-endpoint where brokers split limits), sized below the broker's published limit to leave headroom for retries. Back-pressure upstream — the order router should block on an awaitable token, not spin-retry — so quota exhaustion manifests as latency, not cascading rejections. On approaching the limit, prioritize cancellations over new orders (cancellations reduce risk; new orders increase it). See `backend-fintech-risk-management.md` for why that ordering matters during a volatility event.
+
+## Deep Guidance
+
+### Typical Broker APIs and Their Tradeoffs
+
+Retail/prosumer brokers mostly ship **REST+WebSocket**: REST for order entry and account state, WebSocket for fills, quotes, and order updates. Alpaca, Tradier, Tastytrade, Robinhood (unofficial), and TD Ameritrade/Schwab follow this shape. Latency is 50–500ms per REST call, quotes arrive via WebSocket in tens of ms. Rate limits are HTTP-header-driven (`X-RateLimit-Remaining`, `Retry-After`) and typically 200–1000 requests/minute per key.
+
+**Interactive Brokers** is its own category. The Client Portal Gateway runs locally (or in a sidecar container), maintains a session, and exposes REST on localhost. Sessions expire (roughly every 24h) and require re-auth. IBKR also offers TWS API (native C++/Java/Python bindings over a local socket) and FIX for institutional. The gateway model forces you to treat the broker connection as a stateful component with its own health check and restart policy.
+
+**FIX protocol** (Financial Information eXchange, 4.2 or 4.4 most common) is the institutional standard: persistent TCP session with sequence numbers, heartbeats, and gap-fill recovery. Latency is sub-millisecond but the engineering burden is real: FIX engines (QuickFIX/n, QuickFIX/J, OnixS) require session-state management, store-and-forward semantics, and careful handling of re-sends after disconnect. Use a battle-tested FIX engine; do not roll your own.
+
+**Vendor binary protocols** (some prime brokers, ATS venues) are a further step down the latency/engineering tradeoff — only worth the cost when microseconds matter. For most fintech backends, REST+WS is the right default; FIX is added when institutional routing or smart-order-routing vendors (Trading Technologies, ION, Fidessa) require it.
+
+Reliability and cost also vary. Alpaca's free tier has generous throughput but less uptime history; IBKR has strong reliability but complex gateway management; TD/Schwab is reliable but OAuth is fiddly and the API has frequent deprecations. Track each broker's actual availability via your own monitors — do not trust the status page.
+
+### Authentication Patterns
+
+- **Static API keys (HMAC-signed).** Coinbase, Kraken, older Alpaca. Secret is used to sign a canonical request string (method + path + timestamp + body) with HMAC-SHA256; server verifies. Safer than bearer tokens because the secret never crosses the wire. Rotate quarterly via zero-downtime dual-key windows (new key active; old key accepts for 24h; cut over; revoke old).
+- **Bearer tokens (static).** Tradier, older Alpaca. Simpler; leaked token is used verbatim. Rotate monthly; prefer HMAC when the broker supports both.
+- **OAuth 2.0 with refresh.** TD/Schwab, Robinhood (unofficial), many brokerage aggregators (Plaid Investments, SnapTrade). Access token TTL 15–60 minutes; refresh token TTL days-to-months. Requires a dedicated refresh worker that renews tokens *before* expiry (30–60s buffer) and persists the new access/refresh pair atomically. Refresh-token rotation (where the server issues a new refresh token each time) is common and forces strict single-consumer semantics — only one process may refresh at a time; use a distributed lock (Redis, Postgres advisory lock).
+- **Session-based.** IBKR Client Portal. Login once, keep-alive periodically, reauth on 401. The gateway is stateful; treat it as a singleton per account and health-check the session, not just the gateway process.
+- **mTLS or client certificates.** Institutional FIX sessions and some bank APIs. Certificates rotate annually; automate issuance via your PKI (cert-manager + Vault PKI, AWS Private CA).
+
+Never check secrets into Git. Never pass them on the CLI (they land in shell history and `ps`). Mount them as in-memory files or fetch from the secrets manager at process start. Scrub from logs by field name and by regex (common token shapes: `sk_live_*`, `AK*`, JWT `eyJ*`). Audit-log every credential read with actor id and reason — this is a SOC 2 CC6 control (see `backend-fintech-compliance.md`).
+
+### Adapter Interface Design
+
+Define the contract once, in one place. Below is the normalized TypeScript interface; the same shape translates to Go interfaces, Python Protocols, or protobuf services.
+
+```typescript
+// Normalized broker adapter contract. Every broker-specific class implements
+// this interface; all callers depend on the interface, never on a concrete
+// broker type.
+
+export type OrderSide = 'buy' | 'sell';
+export type OrderType = 'market' | 'limit' | 'stop' | 'stop_limit';
+export type TimeInForce = 'day' | 'gtc' | 'ioc' | 'fok';
+export type OrderStatus =
+  | 'pending_new'      // submitted to us, not yet acked by broker
+  | 'new'              // acked by broker, working
+  | 'partially_filled'
+  | 'filled'
+  | 'canceled'
+  | 'rejected'
+  | 'expired'
+  | 'unknown';         // reconciliation required
+
+export interface PlaceOrderRequest {
+  clientOrderId: string;      // UUID — idempotency key across retries
+  accountId: string;
+  symbol: string;             // normalized ticker (e.g. 'AAPL')
+  side: OrderSide;
+  type: OrderType;
+  quantity: bigint;           // minor units or share count, per asset class
+  limitPrice?: bigint;        // minor units
+  stopPrice?: bigint;
+  timeInForce: TimeInForce;
+  submittedAt: Date;
+}
+
+export interface OrderAck {
+  clientOrderId: string;
+  brokerOrderId: string;
+  status: OrderStatus;
+  acceptedAt: Date;
+}
+
+export interface BrokerAdapter {
+  placeOrder(req: PlaceOrderRequest): Promise<OrderAck>;
+  cancelOrder(clientOrderId: string): Promise<void>;
+  replaceOrder(clientOrderId: string, changes: Partial<PlaceOrderRequest>): Promise<OrderAck>;
+  fetchOrder(clientOrderId: string): Promise<OrderState>;
+  fetchFills(since: Date): Promise<Fill[]>;
+  fetchPositions(accountId: string): Promise<Position[]>;
+  fetchBalance(accountId: string): Promise<Balance>;
+  subscribeFills(onFill: (f: Fill) => void): Promise<Subscription>;
+  subscribeQuotes(symbols: string[], onQuote: (q: Quote) => void): Promise<Subscription>;
+  healthCheck(): Promise<HealthStatus>;
+}
+```
+
+Keep the interface narrow. Broker-specific features (IBKR's advanced order types, Alpaca's fractional shares, crypto venues' margin calls) either fit into optional fields on the normalized types or require a capability flag (`adapter.supports('fractional_shares')`). Resist the temptation to leak broker-specific escape hatches; the moment one caller reaches past the interface, the abstraction stops paying rent.
+
+Version the IDL. Brokers change payloads without notice; when they do, the adapter absorbs it and the interface stays stable — but when *your* normalized shape needs to change (new order type, new status), bump the interface version and support both for a deprecation window.
+
+### Error Harmonization Taxonomy
+
+Every adapter maps broker responses into the internal taxonomy. Get this mapping wrong and you either retry into duplicate orders (worst case) or alert on every transient blip (operational noise). Below is the canonical mapping pattern.
+
+```typescript
+type InternalErrorClass =
+  | 'transient'       // network flake, 5xx; retry with backoff+jitter
+  | 'rate_limited'    // 429; backoff + queue; honor Retry-After
+  | 'invalid'         // 4xx; never retry; surface to user
+  | 'auth'            // 401/403; refresh credential, retry once; alert on repeat
+  | 'outage'          // broker-wide failure; circuit-break; alert on-call
+  | 'unknown';        // indeterminate state; reconcile, do NOT retry
+
+// Alpaca → internal (illustrative).
+// Order matters: ambiguous/specific cases must match BEFORE the generic 5xx
+// branch or a timeout on a mutating call would be silently retried as
+// `transient` and duplicate orders. Split idempotent reads from mutations
+// at the call site so a read-path 500 can still retry freely.
+function classifyAlpacaError(
+  resp: AlpacaErrorResponse,
+  op: 'read' | 'mutate',
+): InternalErrorClass {
+  if (resp.httpStatus === 429) return 'rate_limited';
+  if (resp.httpStatus === 401 || resp.httpStatus === 403) return 'auth';
+  if (resp.httpStatus === 422 && resp.code === 40010001) return 'invalid'; // insufficient buying power
+  if (resp.httpStatus === 422 && resp.code === 40310000) return 'invalid'; // market closed
+  // Specific 5xx shapes first:
+  if (resp.httpStatus === 503 && /maintenance/i.test(resp.message)) return 'outage';
+  if (resp.httpStatus === 504 || resp.code === 'timeout') {
+    // Mutating call + indeterminate response = reconcile, never blind-retry.
+    return op === 'mutate' ? 'unknown' : 'transient';
+  }
+  // Generic 5xx last. A 500 on a mutation is STILL ambiguous — bubble as
+  // `unknown` so the order-lifecycle reconciler resolves it.
+  if (resp.httpStatus >= 500) return op === 'mutate' ? 'unknown' : 'transient';
+  return 'unknown';
+}
+```
+
+The critical distinction is `transient` vs `unknown`. A 500 on a read (fetch positions) is `transient` — retry freely, the read is idempotent. A 500 or timeout on a `placeOrder` call is `unknown` — the order may or may not have reached the broker's matching engine. Retrying risks a duplicate fill; not reconciling risks a missed fill. The correct handling is to enqueue a reconciliation job that polls the broker by `clientOrderId` (which was sent with the request) until the order's fate is determined, then emit the appropriate state transition. This is why `clientOrderId` is mandatory on every place call, not optional. Cross-ref `backend-fintech-order-lifecycle.md` for the full reconciliation flow.
+
+Maintain the error mapping per-broker in a table that product owners and on-call engineers can read without spelunking through code. When the broker ships new error codes (they will, without notice), the table is what gets updated.
+
+### Idempotency With Brokers
+
+Every broker worth integrating accepts a client-supplied order id (`client_order_id`, `clOrdID` in FIX, `externalOrderId`, naming varies). Always send one, always make it a UUID v4 you generate before the first attempt, always persist it before the network call. This gives you two properties: (a) retries dedupe at the broker — a second `placeOrder` with the same `clientOrderId` returns the existing order, not a new one; (b) reconciliation has a stable key to look up on `fetchOrder(clientOrderId)`.
+
+A few brokers reject a second attempt with the same `clientOrderId` even when the first was rate-limited or dropped mid-flight. Your adapter must treat that rejection specifically — it is an *expected* outcome of the retry, not an error. Map it to "fetch the existing order and return the canonical `OrderAck`."
+
+For brokers that do not support client-order-id (rare, mostly legacy or crypto venues), substitute a deduplication window at the adapter: persist `(account, symbol, side, quantity, limit_price, submitted_within_N_seconds)` and reject duplicates before hitting the network. This is weaker than a real idempotency key; flag the broker as a migration priority.
+
+### Broker-Outage Handling
+
+Brokers go down. The 2021 Robinhood and Schwab outages cost real customer money; IBKR's quarterly maintenance windows are announced but often overrun; exchanges halt for SSR or volatility circuit breakers. The adapter must detect and react.
+
+**Circuit breaker per broker per endpoint** (libraries: `opossum` in Node, Resilience4j in Java, `gobreaker` in Go). Trip threshold: e.g. 50% failure rate over 20-call rolling window or 10 consecutive `outage`/`transient` errors. Open state rejects calls immediately with a cached "broker unavailable" response. Half-open probes one request every 30s; success closes the breaker.
+
+Graceful degradation while open: disable new-order submission, continue to allow cancellations (cancellations reduce exposure — prioritize the path that shrinks risk). Continue to serve last-known positions/balances from cache with a clear staleness marker. Page on-call within minutes, not hours — for a brokerage, "we can't place orders" is a customer-impact P0.
+
+User communication: the UI must distinguish "your order failed because of you" from "your order failed because of us." A generic "try again later" is worse than useless during an outage; it creates support load. Publish a status endpoint that the UI reads and surface a specific broker banner.
+
+Record every circuit-breaker state change to the audit log (see `backend-fintech-compliance.md`) — regulators and customers will ask for a postmortem timeline.
+
+### Rate-Limit Strategy: Token Bucket
+
+```typescript
+// Client-side token bucket. One per (broker, credential, endpoint-group).
+// Size the rate below the broker's published limit — leave headroom for
+// retries and for cross-shard contention if multiple workers share credentials.
+
+class TokenBucket {
+  private tokens: number;
+  private lastRefill: number;
+
+  constructor(
+    private readonly capacity: number,       // burst size
+    private readonly refillPerSecond: number // sustained rate
+  ) {
+    this.tokens = capacity;
+    this.lastRefill = Date.now();
+  }
+
+  // Await a token. Resolves when one is available; rejects on timeout.
+  async take(timeoutMs: number = 30_000): Promise<void> {
+    const deadline = Date.now() + timeoutMs;
+    while (true) {
+      this.refill();
+      if (this.tokens >= 1) {
+        this.tokens -= 1;
+        return;
+      }
+      const waitMs = Math.min(
+        (1 / this.refillPerSecond) * 1000,
+        Math.max(0, deadline - Date.now())
+      );
+      if (waitMs <= 0) throw new Error('rate-limit wait timed out');
+      await new Promise((r) => setTimeout(r, waitMs));
+    }
+  }
+
+  private refill(): void {
+    const now = Date.now();
+    const elapsedSec = (now - this.lastRefill) / 1000;
+    this.tokens = Math.min(
+      this.capacity,
+      this.tokens + elapsedSec * this.refillPerSecond
+    );
+    this.lastRefill = now;
+  }
+}
+
+// Usage: await bucket.take() before every broker call. Exhaustion manifests
+// as latency, not errors — upstream request handlers block on the bucket.
+```
+
+For multi-process/multi-region deployments sharing the same credential, a distributed rate-limiter (Redis with Lua scripts, or a dedicated service) replaces the in-memory bucket. Honor the broker's `Retry-After` header as an override — if the broker says "wait 5s," pause the bucket for 5s regardless of its own refill state.
+
+Prioritize traffic when approaching the limit: cancellations before new orders; reads before writes; interactive user requests before background reconciliation jobs. A priority queue in front of the bucket expresses this directly.
+
+### Testing Strategy
+
+- **Sandbox environments.** Alpaca, IBKR, Tradier, and most modern brokers provide paper/sandbox endpoints. Use them for integration tests in CI — not all brokers have them, and sandboxes diverge from production in subtle ways (sandbox may accept orders production would reject for PDT rules, fractional-share limits, etc.). Treat sandbox-green as necessary but not sufficient.
+- **Record/replay for unit tests.** Capture real broker responses (with secrets scrubbed) into fixtures (`nock`, `vcrpy`, `go-vcr`). Unit tests run fully offline and deterministically. Refresh fixtures quarterly and after any broker API version bump.
+- **Contract tests on schedule.** A cron job (hourly or daily) that exercises the full adapter contract against sandbox — place, cancel, fetch, subscribe — and alerts on any behavior drift. Brokers ship breaking changes without notice; contract tests catch them before customers do.
+- **Chaos testing.** Inject rate-limit responses, 5xx, timeouts, and partial payloads into the adapter's HTTP layer and assert the error taxonomy classifies correctly. Tools: `toxiproxy`, `wiremock`, a local test double that replays broker responses.
+- **Reconciliation backfill tests.** Simulate an "unknown" outcome and assert the reconciler converges to the correct terminal state by polling `fetchOrder`. This is where bugs hide — exercise it deliberately.
+
+### Common Pitfalls
+
+- **Retry storms amplifying rate limits.** A broker blip returns 500s; naive retry immediately doubles the traffic, tripping 429s, which retry further. Always use exponential backoff with full jitter (`delay = random(0, base * 2^attempt)`, capped) and a max-attempts limit. Cross-instance coordination via a shared token bucket prevents fleet-wide pile-ons.
+- **Stale auth tokens on long-running processes.** Workers started at 9am with a 60-minute access token stop working at 10am. Refresh proactively with a 30–60s buffer before expiry; run the refresh on a dedicated timer per credential, not inline per request. Persist the refreshed token atomically — two workers refreshing concurrently with a rotating refresh-token scheme will invalidate each other.
+- **Time-zone bugs around market hours.** US equities open 9:30am ET, not 9:30am local. Always store and compute in UTC with an explicit exchange-calendar library (`pandas_market_calendars`, `exchange_calendars`, `NYSE holidays`). Half-days (day after Thanksgiving, Christmas Eve) close at 1:00pm ET — miss these and orders reject late in the day.
+- **Order-state drift when webhooks are missed.** WebSocket disconnects and misses fill events; the internal state lags the broker. Run a periodic `fetchOrder` + `fetchFills` reconciliation (every 30–60s for active orders) even when the WS is healthy. Belt and suspenders — do not trust a single channel.
+- **Assuming broker order-id uniqueness across accounts.** Some brokers scope order ids per account, not globally. Always join on `(broker, account, brokerOrderId)` in your own storage. Your `clientOrderId` is globally unique; the broker's id may not be.
+- **Blindly trusting broker timestamps.** Broker clocks drift. Persist both the broker's timestamp and your own received-at timestamp; reconcile against trading day boundaries using exchange calendars, not broker-reported times.
+- **Silent symbol normalization drift.** `BRK.B` vs `BRK/B` vs `BRK-B` — brokers disagree on tickers with punctuation. Normalize on ingress and egress; keep a per-broker mapping table. Same for options (OCC symbology vs broker-specific shorthand).
+- **Shared credentials across environments.** A staging deploy pointing at production credentials places real orders. Enforce via separate secrets per environment, network-level allowlists, and a pre-flight check that refuses to start if the environment tag does not match the credential's expected tag.
+- **Not testing the sad path under load.** Everything works at 10 req/min in staging; at 500 req/min in production the bucket empties, retries pile up, and the circuit breaker oscillates. Load-test the adapter with realistic traffic shapes including error injection.
+
+See also `backend-fintech-order-lifecycle.md` for the order state machine and reconciliation flows that depend on the adapter's `unknown` classification, `backend-fintech-risk-management.md` for how rate-limit prioritization interacts with kill-switch logic, `backend-fintech-observability.md` for the metrics and traces every adapter should emit, and `backend-fintech-compliance.md` for credential-audit and change-management expectations.

package/content/knowledge/backend/backend-fintech-compliance.md

@@ -0,0 +1,181 @@
+---
+name: backend-fintech-compliance
+description: PCI-DSS, SOC 2, SEC/FINRA regulations for consumer/B2B fintech backends; audit trail immutability; data retention; segregation of duties.
+topics: [backend, fintech, compliance, pci-dss, soc2, sec, finra, audit-trail, gdpr]
+---
+
+Fintech compliance is not a checklist applied at the end — it determines schema design, deployment pipelines, and system boundaries. Most regulations apply based on what a service *touches* (cards, trades, PII), so scope reduction is the single highest-leverage design decision available to engineering. This doc covers the regulatory regimes a typical US/EU fintech encounters, the audit-trail patterns they demand, and concrete implementation choices that keep audits survivable.
+
+## Summary
+
+Which regulations apply depends on what the service handles. Handling card data (PAN, CVV, track data) triggers PCI-DSS v4.0. Storing customer PII, financial records, or operating as a service provider to regulated firms triggers SOC 2 Type II expectations from customers and GLBA obligations (US financial privacy). Executing securities trades or routing orders triggers SEC Rule 17a-4 record retention and FINRA supervisory requirements. Operating in the EU triggers GDPR and — for crypto — MiCA. Serving retail vs institutional customers changes consumer-protection obligations (Reg Z, Reg E, CFPB oversight vs institutional carve-outs).
+
+Compliance cost scales with scope, not with traffic. A service that never sees raw PANs is *out of scope* for most PCI-DSS controls. A microservice that only handles trade metadata (not orders themselves) may be out of the SEC 17a-4 retention perimeter. Practical scope-reduction strategies: tokenize at the edge (Stripe, Marqeta, Basis Theory) so internal services only ever see tokens; keep regulated datastores (ledger, order book, card vault) in dedicated VPCs with narrow IAM; route regulated-data logs to a separate SIEM stream so the main observability stack stays out of scope; design SDK boundaries where PII-sensitive fields are never emitted to general-purpose workers.
+
+Audit trails are the backbone of every fintech regime. Requirements vary but share a common shape: **who did what, to what, when, from where, and with what authorization** — and the log must be tamper-evident and retained for a regulation-specific minimum (SEC 17a-4: 6 years, first 2 easily accessible; SOX: 7 years; GDPR: duration of legitimate-use plus documented retention; PCI-DSS: 1 year minimum, 3 months immediately available). Append-only database tables (enforced via triggers that reject UPDATE/DELETE) satisfy "append-only" but not "tamper-evident" — a DBA with superuser can still alter the table. Tamper-evidence requires either external WORM storage (AWS S3 Object Lock in compliance mode, AWS QLDB, immutable Kafka with retention holds) or cryptographic chaining (each log entry includes the hash of the previous entry, forming a Merkle-like chain verified on read).
+
+Segregation of duties (SoD) is a control, not just good hygiene. For any high-value or irreversible action — wire transfer over a threshold, customer account close, prod DB schema migration, release to production — one human must initiate and a *different* human with appropriate role must approve. SoD failures are the most common SOX/SOC 2 audit findings for fintech engineering orgs. Enforce SoD in the application layer (two-person workflow state machines), in the deploy layer (GitHub required reviewers, protected branches), and in IAM (separate break-glass roles that require another operator to grant). Auditors will ask: "Can the engineer who wrote the code also deploy it to prod? Can the same person initiate and approve a payout?" The answer must be no, with technical enforcement — not policy.
+
+Encryption is assumed baseline: TLS 1.2+ in transit (1.3 preferred), AES-256 at rest via KMS-managed keys (AWS KMS, GCP Cloud KMS, Azure Key Vault, HashiCorp Vault Transit). Field-level encryption for the most sensitive data (SSN, PAN, bank account numbers) in addition to disk encryption, with keys rotated annually. See also `backend-fintech-ledger.md` for double-entry ledger design and `backend-fintech-broker-integration.md` for broker-dealer specific records.
+
+## Deep Guidance
+
+### PCI-DSS Scoping and Tokenization
+
+PCI-DSS v4.0 applies to any system that stores, processes, or transmits cardholder data (CHD) — the Primary Account Number (PAN), cardholder name, expiration date, service code — or sensitive authentication data (full track, CVV, PIN). The cost of compliance is roughly quadratic in the scope of the "cardholder data environment" (CDE): every service inside the CDE needs quarterly ASV scans, annual penetration tests, hardened configurations, FIM, and quarterly access reviews.
+
+**Scope reduction via tokenization** is the dominant pattern. Instead of your application receiving raw PANs, the card is submitted directly from the browser/mobile app to a tokenization provider (Stripe Elements, Braintree Hosted Fields, Marqeta, Basis Theory, Very Good Security) which returns an opaque token. Your backend stores only the token. The card vault is the provider's CDE; your systems are *SAQ A* eligible (the lightest form).
+
+If you must hold PANs directly, consider a dedicated card-vault microservice with its own VPC, its own database, its own deploy pipeline, its own on-call, and narrow IAM so the rest of the org — and most of engineering — is outside the CDE. PAN storage requires strong cryptography with documented key management; display must be truncated (first 6, last 4 at most) unless there's a documented business need.
+
+Never log PANs, CVVs, or track data. Install log-scrubbing middleware that redacts 13–19 digit sequences matching the Luhn check. Same for error reports — configure Sentry/Datadog scrubbing rules and test them.
+
+### SOC 2 Type I vs Type II, and What Engineering Owns
+
+SOC 2 is an attestation framework (not a law) under AICPA TSC (Trust Services Criteria). Type I is a point-in-time assessment of control *design*. Type II assesses *operating effectiveness* over a period (typically 6–12 months). Enterprise customers routinely require Type II before signing.
+
+Trust Services Criteria: **Security (CC1–CC9)** is mandatory; Availability, Confidentiality, Processing Integrity, and Privacy are optional but common. Engineering-touchable criteria:
+
+- **CC6.1–CC6.8 (Logical Access):** IAM, MFA, offboarding, periodic access reviews. Auditors want evidence of MFA enforced, ex-employee access revoked within N days, quarterly access reviews recorded.
+- **CC7.1–CC7.5 (System Operations):** Monitoring, incident response, change detection. Expect to produce alerting config, incident postmortems, and evidence that anomalies were detected and triaged.
+- **CC8.1 (Change Management):** Every production change must be authorized, tested, and reviewed. Auditors will sample production deploys and ask for the associated PR, reviewer approval, test results, and rollback plan.
+- **A1.1–A1.3 (Availability):** Uptime targets, DR tests, backup verification. Run documented DR drills annually.
+
+Automate evidence collection. Tools like Drata, Vanta, Secureframe, Sprinto pull evidence directly from GitHub (PR approvals), AWS (IAM state), Datadog (monitors), HRIS (offboarding). Manual evidence collection at audit time is the most common cause of audit delays.
+
+### SEC Rule 17a-4 and FINRA Supervisory Records
+
+Broker-dealers in the US must retain certain records under SEC Rule 17a-4 (and related 17a-3). Key engineering implications:
+
+- **Retention periods:** Most records — 6 years, with the first 2 years "easily accessible." Customer account records — lifetime of the account plus 6 years. Trade blotters, order tickets, and communications have specific classifications.
+- **Format:** Historically WORM (Write-Once-Read-Many) with audit-system-of-records. The 2022 amendment to 17a-4(f) now permits an "electronic record-keeping system" that uses an audit trail to track and verify changes, as an alternative to strict WORM. Either way: the record must be non-rewriteable, non-erasable, and verifiable.
+- **Third-party access:** You must designate a third party with the ability to download records if the firm becomes unavailable (D3P letter).
+- **Indexing and retrieval:** Records must be indexed and retrievable within a reasonable time for regulatory request.
+
+FINRA Rule 3110 imposes supervisory obligations: written supervisory procedures (WSPs), review of correspondence, review of trade exceptions. Engineering typically supports these via queryable audit logs and exception-reporting pipelines.
+
+Common implementation: trade-event stream written to AWS S3 with Object Lock in **compliance mode** (not governance — compliance mode cannot be disabled even by root) with a 6-year retention period, fed by an immutable Kafka topic, with a parallel queryable index (OpenSearch, Postgres read model) for retrieval. See also `backend-fintech-ledger.md` and `backend-fintech-order-lifecycle.md`.
+
+### Immutable Audit Log Patterns
+
+**Pattern 1: Append-only table with trigger-enforced immutability (Postgres).**
+
+```sql
+CREATE TABLE audit_log (
+  id            BIGSERIAL PRIMARY KEY,
+  occurred_at   TIMESTAMPTZ NOT NULL DEFAULT now(),
+  actor_id      UUID NOT NULL,
+  actor_type    TEXT NOT NULL,         -- 'user' | 'system' | 'api_key'
+  action        TEXT NOT NULL,         -- 'order.submit' | 'account.close'
+  resource_type TEXT NOT NULL,
+  resource_id   TEXT NOT NULL,
+  request_id    UUID NOT NULL,
+  ip_address    INET,
+  user_agent    TEXT,
+  payload       JSONB NOT NULL,
+  prev_hash     BYTEA NOT NULL,        -- hash of previous row's row_hash
+  row_hash      BYTEA NOT NULL         -- hash of this row's content + prev_hash
+);
+
+CREATE OR REPLACE FUNCTION audit_log_immutable()
+RETURNS trigger LANGUAGE plpgsql AS $$
+BEGIN
+  RAISE EXCEPTION 'audit_log is append-only';
+END;
+$$;
+
+CREATE TRIGGER audit_log_no_update BEFORE UPDATE ON audit_log
+  FOR EACH ROW EXECUTE FUNCTION audit_log_immutable();
+CREATE TRIGGER audit_log_no_delete BEFORE DELETE ON audit_log
+  FOR EACH ROW EXECUTE FUNCTION audit_log_immutable();
+
+REVOKE UPDATE, DELETE, TRUNCATE ON audit_log FROM PUBLIC;
+```
+
+Triggers prevent UPDATE/DELETE via the application path. A DBA with BYPASSRLS or superuser can still mutate — which is why tamper-evidence needs cryptographic chaining or external WORM.
+
+**Pattern 2: Hash-chain computation for tamper evidence.**
+
+```typescript
+import { createHash } from 'node:crypto';
+
+function rowHash(prevHash: Buffer, row: AuditRow): Buffer {
+  const canonical = JSON.stringify({
+    occurredAt: row.occurredAt.toISOString(),
+    actorId: row.actorId,
+    action: row.action,
+    resourceType: row.resourceType,
+    resourceId: row.resourceId,
+    payload: row.payload, // must be canonicalized (sorted keys)
+  });
+  return createHash('sha256').update(prevHash).update(canonical).digest();
+}
+
+// Verification: walk the chain, recompute each hash, compare to stored row_hash.
+// Any mutation anywhere in the chain invalidates all subsequent hashes.
+// Publish the latest root hash daily (to an external WORM store, or public
+// notary like OpenTimestamps) so even a full-DB rewrite is detectable.
+```
+
+**Pattern 3: External WORM.** AWS QLDB provides a managed ledger with cryptographic verification built in (but note AWS announced QLDB end-of-support in 2025; current guidance is Aurora PostgreSQL with application-layer chaining). Immutable Kafka (with compaction disabled and retention set to forever) plus S3 Object Lock in compliance mode is the more durable path today.
+
+### Tokenization and Scope-Reduction Boundary
+
+```text
+// Scope-reduction sequence — card capture via hosted fields
+//
+//   Browser                Payment Provider            Your Backend
+//     |                        (CDE)                      (out of CDE)
+//     |----- PAN + CVV ------->|                           |
+//     |                        |-- vault + tokenize ------>|
+//     |                        |                           |
+//     |<-- token (tok_abc) ----|                           |
+//     |------------ token ---------------------------------|
+//     |                                                    |
+//     |                                  [store token, last4, brand]
+//     |                                  [never sees raw PAN]
+```
+
+The boundary is enforced by: (a) hosted fields JavaScript that POSTs directly to the provider, (b) CSP and network rules that prevent your backend from reaching card-network endpoints directly, (c) log scrubbing, (d) an SAQ A attestation that documents you never touch CHD.
+
+### Encryption-at-Rest and In-Transit Expectations
+
+- **In transit:** TLS 1.2 minimum (1.3 preferred). Internal service-to-service: mutual TLS or a service mesh (Istio, Linkerd, Consul Connect) that provides mTLS. No plaintext — even on private networks — for regulated data.
+- **At rest:** Disk encryption (EBS, RDS, S3 SSE) is the baseline but not sufficient for the most sensitive data. Add field-level encryption for SSN, PAN, bank account numbers, MFA seeds. Use envelope encryption: data encryption keys (DEKs) wrap each record, KMS holds the key-encryption key (KEK). AWS KMS, GCP Cloud KMS, Azure Key Vault, or HashiCorp Vault Transit all support this pattern.
+- **Key rotation:** Annual minimum for KEKs. DEKs rotate via re-encryption on access patterns. Document the rotation runbook — auditors will ask.
+- **Backups:** Encrypted with the same rigor as primary data. Test restore quarterly, document the result.
+
+### Data Residency and Localization
+
+EU GDPR, UK GDPR, India DPDP, Brazil LGPD, and increasingly US state laws (California CCPA/CPRA, Colorado CPA) impose data-locality and cross-border-transfer constraints. Design choices:
+
+- **Region-per-tenant:** A tenant lives in exactly one region; all their data (primary DB, backups, caches, search indexes, logs) stays in that region. Route requests by tenant → region mapping at the edge (Cloudflare Workers, AWS Global Accelerator, route-53 geolocation).
+- **Partition keys include region:** So you can evacuate a region by range-scanning the partition map.
+- **No global secondary indexes that span regions.** That includes search indexes, analytics warehouses, and audit aggregators.
+- **Document your data-flow map.** GDPR Article 30 records of processing activities require it. Keep it in-repo and generated from code where possible (schema annotations → flow map).
+
+Retrofitting region isolation after launch is 10x the cost of building it in. If there's any chance you'll serve EU customers, design for it on day one.
+
+### Change-Management Evidence (SOC 2 CC8.1)
+
+Your CI/CD pipeline is auditor-facing. For every production deploy, auditors want to see:
+
+- **Artifact provenance:** The exact commit SHA, the container digest (sha256), the builder identity. SLSA Level 2+ provenance (via GitHub Actions OIDC, Sigstore cosign) makes this cryptographically verifiable.
+- **Approval trail:** PR with required reviewers, required status checks (tests, security scans), branch protection enforced. No direct pushes to main. GitHub's audit log captures this.
+- **Test evidence:** Test results archived per deploy (JUnit XML, coverage report) and tied to the deploy via the commit SHA.
+- **Rollback capability:** Documented procedure, tested in production at least annually. Include automated rollback on health-check failure.
+- **Deploy record:** Who triggered, what artifact, when, to what environment, what configuration changes. Deploy tools (ArgoCD, Spinnaker, GitHub Deployments) produce this natively.
+
+Separate the CI identity from the CD identity. The CI identity can build and sign; only the CD identity (with stricter controls) can deploy to prod. This is SoD at the pipeline layer.
+
+### Known Pitfalls
+
+- **Secrets in logs.** A developer logs `console.log(req.headers)` including `Authorization`. Prevention: structured-logging middleware with an allowlist of fields, deny-list of headers (Authorization, Cookie, X-API-Key), unit tests asserting redaction.
+- **Debug-mode leaks.** Stack traces, `DEBUG=*`, SQL echo — off in production. Verify via a smoke test in the deploy pipeline.
+- **PII in error reports.** Sentry/Datadog by default captures request bodies. Configure beforeSend/scrubbing for known PII field names; validate with synthetic PII in staging.
+- **Background workers bypassing request-context audit hooks.** A cron job that updates account balances without writing to `audit_log` creates a compliance gap. Audit-log writes must live in the domain layer (on the entity mutation), not in HTTP middleware.
+- **Shadow datastores.** Read replicas, BI warehouses, and ML feature stores often get regulated data without inheriting the controls of the primary store. Every copy of the data must inherit the same retention, access, and encryption rules.
+- **Third-party SaaS with unaudited access.** Customer-support tools (Intercom, Zendesk), analytics (Segment, Amplitude), and AI assistants can exfiltrate PII. Gate every new SaaS through a vendor-review process with a DPA.
+- **Employee access without audit.** Support staff viewing customer accounts must log every view (access-log table, reason code). This is a common SOC 2 finding and GLBA Safeguards Rule expectation.
+
+See also `backend-fintech-ledger.md`, `backend-fintech-testing.md`, and `backend-fintech-observability.md`.