sea-dev 1.0.0

package/docs/prd-form-builder-strategy.md

@@ -0,0 +1,1120 @@
+# PRD: Intelligent Form Builder Strategy
+
+## Executive Summary
+
+This PRD outlines improvements to the form creation experience to better serve users building data extraction workflows. The core opportunity is transforming form creation from an isolated, manual task into an intelligent, workflow-aware process that maximizes AI extraction accuracy.
+
+---
+
+## 1. Jobs to Be Done (JTBD) Analysis
+
+### Primary JTBD
+
+> **"When I'm setting up a new underwriting workflow, I want to quickly define what data I need extracted from documents, so I can start processing deals without manual data entry."**
+
+### Secondary JTBDs
+
+| JTBD | Current Gap |
+|------|-------------|
+| "I want to extract bank statement transactions" | User must manually know to create collection groups; no bank statement template |
+| "I want the AI to assess/calculate values after extraction" | Derived fields exist but chat builder cannot create them |
+| "I want related forms to work together in a workflow" | Each form created in isolation with no workflow context |
+| "I want to track changes to my forms over time" | No version history; only chat history exists |
+| "I want to reuse form patterns across workflows" | Clone exists but no template library or form references |
+
+---
+
+## 2. Current System Architecture
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                         WORKFLOW                                 │
+│  ┌─────────┐  ┌─────────┐  ┌─────────┐                          │
+│  │ Form A  │  │ Form B  │  │ Form C  │  ← Created independently │
+│  │ (chat)  │  │ (chat)  │  │ (chat)  │    No shared context     │
+│  └────┬────┘  └────┬────┘  └────┬────┘                          │
+└───────┼────────────┼────────────┼───────────────────────────────┘
+        │            │            │
+        ▼            ▼            ▼
+   ┌─────────────────────────────────────┐
+   │         FORM STRUCTURE              │
+   │  ┌──────────────────────────┐       │
+   │  │ Group (multi: false)     │       │
+   │  │  ├─ Field (8 types)      │       │
+   │  │  ├─ Field                │       │
+   │  │  └─ Derived* ← NOT IN CHAT       │
+   │  ├──────────────────────────┤       │
+   │  │ Collection (multi: true) │       │
+   │  │  ├─ Field (pinned first) │       │
+   │  │  └─ Field                │       │
+   │  └──────────────────────────┘       │
+   └─────────────────────────────────────┘
+                    │
+                    ▼
+   ┌─────────────────────────────────────┐
+   │         DATA EXTRACTION             │
+   │  • PDF/Document upload              │
+   │  • Layout analysis                  │
+   │  • Field hint-guided extraction     │
+   │  • Derived field LLM assessment     │
+   └─────────────────────────────────────┘
+```
+
+### Current Chat Builder Limitations
+
+| Feature | In Schema | Chat Can Create |
+|---------|-----------|-----------------|
+| Groups | ✅ | ✅ |
+| Collection Groups (multi) | ✅ | ✅ |
+| 8 Field Types | ✅ | ✅ |
+| Field Hints | ✅ | ✅ |
+| **Derived Fields** | ✅ | ❌ |
+| **Bank Statement Type** | ✅ | ❌ |
+| **Workflow Context** | N/A | ❌ |
+| **Other Forms Awareness** | N/A | ❌ |
+| **Submission-Aware Editing** | N/A | ❌ (fails silently) |
+
+---
+
+## 3. Opportunity Analysis
+
+### What AI Extraction is Good At
+
+- Extracting text/numbers visible in documents
+- Identifying repeating patterns (transaction rows, periods)
+- Matching values to clearly-hinted fields
+- Inferring structure from document layout
+
+### What Doesn't Make Sense (Current Limitations)
+
+- External data source integration (credit scores, APIs)
+- Multi-document aggregation/analytics
+- Running balance calculations across years
+- Data that isn't in an uploadable document
+
+### Key Insight
+
+> **"Everything should be conceivably available in an uploaded document"**
+
+---
+
+## 4. Proposed Improvements
+
+### 4.1 Quick Wins (Low Effort, High Impact)
+
+#### A. Enable Derived Fields in Chat Builder
+
+**Current:** Chat schema omits `derived` field array
+
+**Fix:** Add derived fields to `formGroupSchema`
+
+```typescript
+const formGroupSchema = z.object({
+  // ... existing fields
+  derived: z.array(z.object({
+    id: z.string().optional(),
+    name: z.string(),
+    prompt: z.string().describe("LLM prompt for post-extraction assessment"),
+  })).optional(),
+});
+```
+
+**Value:** Unlocks assessment fields (e.g., "Is this borrower high-risk based on extracted data?")
+
+#### B. Enhance System Prompt with Form Knowledge
+
+**Current prompt knows:**
+
+- Form name, workflow name
+- Basic CRUD instructions
+
+**Add:**
+
+- Explanation of collection groups (multi=true) and when to use them
+- Derived fields concept and examples
+- Field type best practices (money vs number, date formatting)
+- Document-extractability constraint
+- First-field-pinned UX for collections
+
+#### C. Add Smart Defaults for Common Patterns
+
+When user says "bank statement" → suggest:
+
+```
+Collection Group: Transactions
+  - Date (pinned first)
+  - Description
+  - Amount (money)
+  - Running Balance (money)
+```
+
+#### D. Expose Bank Statement Group Type
+
+**Current:** `type: "bank_statement"` exists in schema but not exposed in chat
+
+**Fix:** Allow chat to set group type for specialized extraction
+
+#### G. Extraction Guidance in Prompts
+
+**Effort: S | Impact: High**
+
+Add extraction-quality hints to system prompt:
+
+```markdown
+## Form Design Best Practices for AI Extraction
+
+1. **Be specific in field hints** - "Net income after tax from line 28" > "Net income"
+2. **Use collection groups for repeating data** - Transactions, periods, line items
+3. **Pin the key identifier first** - Date for time series, Account # for statements
+4. **Match document terminology** - Use terms that appear in the actual document
+5. **Derived fields for assessment** - Use these for judgments, not extraction
+6. **Only extractable data** - Everything must be conceivably in an uploaded document
+```
+
+#### K. Submission-Aware Edit Restrictions
+
+**Effort: S | Impact: High**
+
+**Problem:** When a form has existing submissions (populated data), users cannot delete fields or groups without breaking data integrity. Currently, the chat builder is unaware of this constraint and allows users to attempt destructive edits, which fail silently with a confusing error.
+
+**Current behavior:**
+- User asks chat to "remove the Revenue field"
+- Chat attempts `persistForm()` with field removed
+- Backend rejects the change (foreign key constraint)
+- User sees generic error with no explanation
+
+**Solution:** Add `fetchFormEditability()` tool that chat calls before any edit
+
+```typescript
+// New tool for chat builder
+function fetchFormEditability({ formId }) {
+  return tool({
+    description: "Check what edits are allowed on this form based on existing submissions",
+    execute: async () => {
+      const submissions = await getSubmissionCountForForm(formId);
+      const hasSubmissions = submissions > 0;
+
+      return {
+        hasSubmissions,
+        submissionCount: submissions,
+        restrictions: hasSubmissions ? {
+          canDeleteFields: false,
+          canDeleteGroups: false,
+          canChangeFieldTypes: false,
+          canRenameFields: true,
+          canUpdateHints: true,
+          canAddFields: true,
+          canAddGroups: true,
+          canReorderFields: true,
+        } : {
+          // All edits allowed for forms without submissions
+          canDeleteFields: true,
+          canDeleteGroups: true,
+          canChangeFieldTypes: true,
+          canRenameFields: true,
+          canUpdateHints: true,
+          canAddFields: true,
+          canAddGroups: true,
+          canReorderFields: true,
+        },
+        message: hasSubmissions
+          ? `This form has ${submissions} submission(s). You can add fields, rename fields, and update hints, but cannot delete fields or groups.`
+          : "This form has no submissions. All edits are allowed."
+      };
+    }
+  });
+}
+```
+
+**System prompt addition:**
+
+```markdown
+## Edit Restrictions for Forms with Data
+
+Before making any changes to a form, you MUST call `fetchFormEditability()` to check what edits are allowed.
+
+If the form has existing submissions:
+- ✅ **Allowed:** Add new fields, add new groups, rename fields, update hints, reorder fields
+- ❌ **Not allowed:** Delete fields, delete groups, change field types
+
+If a user requests a restricted action, explain:
+> "This form has [N] submissions with data. Deleting fields would break existing data integrity.
+> You can: add new fields, rename existing fields, or update field hints.
+> To delete fields, you would need to first delete all submissions or clone this form."
+```
+
+**Value:**
+- Prevents confusing errors
+- Educates users about data integrity constraints
+- Suggests valid alternatives
+
+---
+
+### 4.2 Medium Projects
+
+#### E. Workflow-Aware Form Creation
+
+**Problem:** User creates Form A, then Form B, but B has no context of A
+
+**Solution:** New `fetchWorkflowContext()` tool
+
+```typescript
+{
+  workflow: { name, hint },
+  existingForms: [
+    { name: "Income Statement", groups: [...], purpose: "..." },
+    { name: "Balance Sheet", groups: [...] }
+  ],
+  commonPatterns: ["All forms have a 'Document Date' field"]
+}
+```
+
+**Prompt enhancement:**
+
+> "Consider how this form relates to existing forms in the workflow. Ensure consistent naming conventions and complementary data structures."
+
+#### F. Form Templates Library
+
+Create reusable templates for common document types:
+
+- Bank Statement (transactions collection)
+- Income Statement (line items by period)
+- Balance Sheet (assets, liabilities sections)
+- Tax Return (key fields)
+
+**Implementation:** Template selector in form creation modal + template-aware chat
+
+---
+
+### 4.3 Bigger Projects
+
+#### H. Version Control & History
+
+**Schema additions:**
+
+```typescript
+// form_version table
+form_version {
+  id: string
+  formId: string (FK)
+  version: number
+  snapshot: jsonb  // Full form structure at this version
+  changedBy: string (FK to member)
+  changeNote: string
+  createdAt: timestamp
+}
+
+// form_reference table (for lineage)
+form_reference {
+  id: string
+  sourceFormId: string (FK)
+  targetFormId: string (FK)
+  referenceType: enum ["derived_from", "related_to", "supersedes"]
+  createdAt: timestamp
+}
+```
+
+**Features:**
+
+- View version history (Google Docs-style timeline)
+- Compare versions (diff view)
+- Restore previous version
+- Track who changed what and why
+- Reference chains between forms
+
+#### I. Cross-Form Intelligence & Lineage Tracking
+
+**UUID-Based Lineage System:**
+
+Every entity gets a stable UUID for full traceability:
+
+```typescript
+// Field-level lineage
+form_field {
+  uuid: string  // Immutable identifier
+  // ... existing fields
+}
+
+// Extraction-level lineage
+submission_item_version {
+  uuid: string  // Unique per extraction attempt
+  formFieldUuid: string  // Links to field at extraction time
+  formVersionId: string  // Which form version was used
+  extractedAt: timestamp
+  // ... existing fields
+}
+
+// Point-in-time tracking
+extraction_snapshot {
+  id: string
+  submissionId: string
+  formVersionId: string  // Form state at extraction time
+  extractedAt: timestamp
+  documentHash: string   // Source document fingerprint
+}
+```
+
+**Cross-Form Join Keys:**
+
+New field type: `identifier` - enables entity resolution across forms
+
+```typescript
+// Schema addition
+formFieldType: [...existing, "identifier"]
+
+// identifier field config
+identifier_config {
+  fieldId: string
+  entityType: enum ["company", "person", "account", "transaction", "custom"]
+  matchStrategy: enum ["exact", "fuzzy", "normalized"]
+  canonicalSource: string (FK to form)  // Optional: which form is source of truth
+}
+
+// Resolution table
+entity_resolution {
+  id: string
+  entityType: string
+  canonicalUuid: string  // Master entity ID
+  sourceFieldUuid: string
+  sourceValue: string
+  confidence: number  // For fuzzy matches
+  resolvedAt: timestamp
+}
+```
+
+**How it works:**
+
+1. User marks a field as `identifier` type (e.g., "Company Name" in Form A)
+2. When same entity appears in Form B, system attempts resolution:
+   - **Exact match:** Same normalized value → auto-link
+   - **Fuzzy match:** "Acme Corp" ↔ "ACME Corporation" → suggest link with confidence score
+   - **Manual resolution:** User confirms/rejects suggested matches
+3. Linked entities share a `canonicalUuid` enabling cross-form queries
+
+**Use cases:**
+
+- Join bank statement transactions to income statement line items by date
+- Link company data across multiple document types
+- Track same borrower across deals
+- Reconciliation: flag when same entity has conflicting values
+
+#### J. Workflow Builder Mode
+
+New experience for creating entire workflows:
+
+1. User describes workflow goal ("Commercial loan underwriting")
+2. AI suggests complete form set with relationships
+3. User refines holistically before generating individual forms
+4. Forms created with cross-references and consistent conventions
+
+---
+
+## 5. Data Architecture for Version Control
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                     VERSION CONTROL MODEL                        │
+│                                                                  │
+│  Form ─────┬──── FormVersion (1, 2, 3...)                       │
+│            │         └── snapshot: { groups, fields, derived }   │
+│            │         └── changeNote, changedBy, timestamp        │
+│            │                                                     │
+│            └──── FormReference                                   │
+│                      └── sourceForm → targetForm                 │
+│                      └── type: derived_from | related_to         │
+│                                                                  │
+│  View History: Timeline of versions with diffs                   │
+│  Restore: Create new version from historical snapshot            │
+│  Lineage: Graph of form relationships                            │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+---
+
+## 6. Prioritized Roadmap
+
+| Priority | Item | Effort | Impact | Dependencies |
+|----------|------|--------|--------|--------------|
+| **P0** | A. Derived fields in chat | S | High | None |
+| **P0** | B. Enhanced system prompt | S | High | None |
+| **P0** | G. Extraction guidance in prompts | S | High | B |
+| **P0** | K. Submission-aware edit restrictions | S | High | None |
+| **P1** | C. Smart defaults for patterns | M | Medium | B |
+| **P1** | D. Bank statement group type | S | Medium | None |
+| **P1** | E. Workflow-aware form creation | M | High | None |
+| **P2** | F. Form templates library | M | Medium | C |
+| **P3** | H. Version control & lineage UUIDs | L | High | Schema migration |
+| **P3** | I. Cross-form intelligence + identifier field type | L | High | H, E |
+| **P4** | J. Workflow builder mode | XL | Very High | E, F, I |
+
+---
+
+## 7. Success Metrics
+
+| Metric | Current | Target |
+|--------|---------|--------|
+| Forms with derived fields | Unknown (low) | 50%+ of forms |
+| Chat-created vs manual forms | Unknown | 80%+ via chat |
+| Time to first working form | Unknown | < 5 minutes |
+| Extraction accuracy | Baseline TBD | +20% improvement |
+| Form revision frequency | Unknown | -30% (better first drafts) |
+
+---
+
+## 8. Technical Debt & Maintainability Notes
+
+1. **Prompt versioning** - Consider storing prompt versions to correlate form quality with prompt changes
+2. **Schema migrations** - Version control tables need careful migration strategy
+3. **Backwards compatibility** - FormVersion snapshots must handle schema evolution
+4. **Chat token limits** - Workflow context could bloat prompts; need truncation strategy
+5. **Template maintenance** - Templates need regular updates as extraction improves
+
+---
+
+## 9. Open Questions
+
+1. Should derived fields support referencing other forms, or only current form data?
+2. Version control: every save, or explicit "checkpoint" UX?
+3. Should templates be org-specific, global, or both?
+4. How to handle form references when source form is deleted?
+5. Should workflow builder be a separate mode or integrated into existing flow?
+
+---
+
+## 10. Next Steps
+
+1. **Validate priorities** - Confirm P0 items align with roadmap
+2. **Implement quick wins** - A, B, D, G can be done immediately
+3. **Scope version control** - Decide on UX (timeline view, diff UI) before schema design
+4. **User research** - What patterns do users struggle with most?
+
+---
+
+## 11. Future Ideas & Extended Scope
+
+The following ideas represent potential future enhancements that extend beyond the current roadmap. They are documented here for consideration in later phases.
+
+---
+
+### 11.1 Capability Model as Formal System
+
+**Concept:** Instead of embedding extractability rules in prompts, create a formal internal "capability profile" that the builder validates against programmatically.
+
+**Capability Profile Structure:**
+
+```typescript
+interface CapabilityProfile {
+  version: string;
+
+  extraction: {
+    // What can be extracted
+    singleDocumentFields: true;
+    tabularStructures: true;
+    textClassification: true;
+    textSummarization: true;
+
+    // What cannot (yet) be extracted
+    multiDocumentAggregation: false;
+    externalApiIntegration: false;
+    crossYearAnalytics: false;
+    realTimeDataFetch: false;
+  };
+
+  fieldPatterns: {
+    supported: ["direct_value", "table_cell", "derived_from_text"];
+    unsupported: ["api_lookup", "multi_doc_aggregate", "time_series_calc"];
+  };
+}
+```
+
+**Behaviour:**
+
+- Chat builder queries capability profile before suggesting fields
+- Warns user: "This field requires multi-document aggregation which is not yet supported"
+- Tags fields as `future_capability: true` for later enablement
+- Profile is versioned and updated as extraction engine improves
+
+**Value:** Prevents invalid forms at creation time; provides clear upgrade path as capabilities expand.
+
+**Effort:** Medium | **Dependencies:** None
+
+---
+
+### 11.2 Form Metadata Schema
+
+**Concept:** Enrich forms with structured metadata that informs extraction, templates, and discoverability.
+
+**Schema Additions:**
+
+```typescript
+form {
+  // ... existing fields
+
+  // New metadata fields
+  docType: enum ["bank_statement", "income_statement", "balance_sheet",
+                 "tax_return", "registry_document", "invoice", "custom"]
+
+  intendedSources: enum[] ["pdf", "html", "ocr_scan", "structured_upload"]
+
+  description: string  // Human-readable purpose
+
+  exampleDocsUrl: string[]  // Links to sample documents
+
+  extractionNotes: string  // Tips for the extraction engine
+
+  industry: enum ["fintech", "real_estate", "sme_lending", "generic"]
+}
+```
+
+**Use Cases:**
+
+- Template library filters by `docType` and `industry`
+- Extraction engine uses `intendedSources` to optimize processing
+- `exampleDocsUrl` enables test harness and onboarding
+- Search/filter forms by metadata
+
+**Effort:** Small | **Dependencies:** None
+
+---
+
+### 11.3 Version Status Workflow
+
+**Concept:** Replace simple linear versioning with an explicit state machine for governance.
+
+**States:**
+
+```
+draft → active → archived
+          ↑
+          └── (restore creates new draft)
+```
+
+**Rules:**
+
+- Only ONE `active` version per form at any time
+- Workflows can ONLY reference `active` versions
+- `draft` versions are work-in-progress, not used in extraction
+- `archived` versions are read-only historical records
+- Restoring an old version creates a NEW draft (never mutates history)
+
+**Schema:**
+
+```typescript
+form_version {
+  // ... existing fields
+  status: enum ["draft", "active", "archived"]
+  publishedAt: timestamp | null
+  publishedBy: string (FK) | null
+}
+```
+
+**UI Implications:**
+
+- "Publish" button to promote draft → active
+- Warning when publishing: "This will replace the current active version"
+- Version list shows status badges
+- Filter by status in version history
+
+**Trade-offs:**
+
+- More governance = more friction for quick iterations
+- Consider: auto-publish for minor changes, require explicit publish for breaking changes?
+
+**Effort:** Medium | **Dependencies:** Version control (H)
+
+---
+
+### 11.4 Role-Based Access Control for Forms
+
+**Concept:** Form-specific permissions beyond organization-level access.
+
+**Roles:**
+
+| Role | Capabilities |
+|------|-------------|
+| **Viewer** | Read forms, view history, view submissions |
+| **Editor** | Create/edit forms, create draft versions |
+| **Publisher** | Promote drafts to active, rollback versions |
+| **Admin** | Delete forms, manage form permissions, bulk operations |
+
+**Schema:**
+
+```typescript
+form_permission {
+  id: string
+  formId: string (FK)
+  memberId: string (FK)
+  role: enum ["viewer", "editor", "publisher", "admin"]
+  grantedBy: string (FK)
+  grantedAt: timestamp
+}
+```
+
+**Inheritance:**
+
+- Organization admins have implicit `admin` on all forms
+- Workflow owners have implicit `publisher` on workflow forms
+- Default role for org members: configurable (viewer/editor)
+
+**Effort:** Medium | **Dependencies:** Existing org/member model
+
+---
+
+### 11.5 Audit Logging
+
+**Concept:** Comprehensive logging of all form-related actions for compliance and debugging.
+
+**Events to Log:**
+
+```typescript
+form_audit_log {
+  id: string
+  formId: string (FK)
+  formVersionId: string (FK) | null
+
+  action: enum [
+    "form_created",
+    "form_updated",
+    "form_deleted",
+    "version_created",
+    "version_published",
+    "version_archived",
+    "version_restored",
+    "permission_granted",
+    "permission_revoked"
+  ]
+
+  actorId: string (FK to member)
+  actorType: enum ["user", "system", "api"]
+
+  metadata: jsonb  // Action-specific details
+
+  ipAddress: string | null
+  userAgent: string | null
+
+  timestamp: timestamp
+}
+```
+
+**Query Patterns:**
+
+- "Show me all changes to Form X in the last 30 days"
+- "Who published version 5?"
+- "What did user Y change across all forms?"
+- "When was this field added?"
+
+**Retention:** Consider 90-day hot storage, archive to cold storage thereafter.
+
+**Effort:** Small-Medium | **Dependencies:** None (can be added independently)
+
+---
+
+### 11.6 Form Version Test Harness
+
+**Concept:** CI/CD-style testing for form versions before promotion to active.
+
+**Workflow:**
+
+```
+1. Create/edit form → draft version created
+2. Upload test documents (or use existing samples)
+3. Run extraction against draft version
+4. Compare results to:
+   - Expected values (if defined)
+   - Previous version results (regression detection)
+5. Generate quality report:
+   - Field-level accuracy scores
+   - Confidence distributions
+   - Extraction failures/warnings
+6. Human review → publish or iterate
+```
+
+**Schema:**
+
+```typescript
+form_test_run {
+  id: string
+  formVersionId: string (FK)
+
+  status: enum ["pending", "running", "completed", "failed"]
+
+  testDocuments: string[]  // Document IDs used
+
+  results: jsonb {
+    fieldAccuracy: { [fieldId]: number }
+    overallScore: number
+    regressions: { field, previous, current, delta }[]
+    warnings: string[]
+  }
+
+  comparedToVersionId: string (FK) | null  // For regression testing
+
+  runAt: timestamp
+  runBy: string (FK)
+}
+```
+
+**UI:**
+
+- "Test this version" button in form editor
+- Test results panel with pass/fail indicators
+- Side-by-side comparison with previous version
+- Block publish if score below threshold (configurable)
+
+**Effort:** Large | **Dependencies:** Version control (H), sample document storage
+
+---
+
+### 11.7 Change Impact Analysis
+
+**Concept:** Understand downstream effects before making form changes.
+
+**Questions Answered:**
+
+- "If I remove this field, which workflows are affected?"
+- "Which submissions used this version?"
+- "What reports/dashboards reference this field?"
+- "Are there derived fields that depend on this?"
+
+**Implementation:**
+
+```typescript
+// Dependency graph
+form_dependency {
+  id: string
+  sourceType: enum ["field", "group", "form", "workflow"]
+  sourceId: string
+  targetType: enum ["field", "derived_field", "workflow_step", "report"]
+  targetId: string
+  dependencyType: enum ["references", "derives_from", "displays"]
+}
+
+// Impact analysis query
+function analyzeImpact(formId, proposedChanges) {
+  return {
+    affectedWorkflows: Workflow[],
+    affectedSubmissions: { count, dateRange },
+    brokenDependencies: Dependency[],
+    derivedFieldsImpacted: DerivedField[],
+    severity: "low" | "medium" | "high" | "breaking"
+  }
+}
+```
+
+**UI:**
+
+- Impact preview modal before publishing changes
+- Warning badges on fields with dependencies
+- "Breaking change" confirmation for high-severity impacts
+
+**Effort:** Large | **Dependencies:** Cross-form intelligence (I), version control (H)
+
+---
+
+### 11.8 Environment Separation
+
+**Concept:** Separate dev/staging/prod environments for forms and workflows.
+
+**Model:**
+
+```
+Organization
+  └── Environment (dev, staging, prod)
+        └── Forms (isolated per environment)
+        └── Workflows (isolated per environment)
+        └── Submissions (isolated per environment)
+```
+
+**Promotion Flow:**
+
+```
+dev → staging → prod
+  │       │
+  └───────┴── Copy form version between environments
+              (not move - each env has own instance)
+```
+
+**Schema:**
+
+```typescript
+environment {
+  id: string
+  organizationId: string (FK)
+  name: string  // "dev", "staging", "prod", or custom
+  type: enum ["development", "staging", "production"]
+  isDefault: boolean
+}
+
+// All major entities get environment FK
+form {
+  // ... existing
+  environmentId: string (FK)
+}
+```
+
+**Features:**
+
+- Environment switcher in UI
+- Promote form version: copies to target environment
+- Environment-specific API keys
+- Production protection: require approval for changes
+
+**Trade-offs:**
+
+- Significant infrastructure complexity
+- Data isolation questions (can prod reference staging data?)
+- May be overkill for smaller teams
+
+**Effort:** Large | **Dependencies:** Version control (H)
+
+---
+
+### 11.9 Enhanced DerivedFieldConfig
+
+**Concept:** Richer configuration for derived/assessment fields beyond simple prompts.
+
+**Extended Schema:**
+
+```typescript
+derived_field_config {
+  id: string
+  formDerivedId: string (FK)
+
+  // Dependencies
+  dependsOnFieldIds: string[]  // Explicit field references
+  dependsOnGroupIds: string[]  // Entire group as context
+  crossFormDependencies: {     // v2+: cross-form references
+    formId: string
+    fieldIds: string[]
+  }[]
+
+  // AI behavior
+  aiMode: enum [
+    "classification",    // Categorize into predefined options
+    "summarization",     // Generate text summary
+    "scoring",           // Numeric score (0-100, 1-5, etc.)
+    "boolean_flag",      // Yes/No determination
+    "extraction",        // Extract specific value from context
+    "calculation"        // Compute from other fields (with formula)
+  ]
+
+  // Mode-specific config
+  classificationOptions: string[] | null  // For classification mode
+  scoringRange: { min: number, max: number } | null
+  calculationFormula: string | null  // e.g., "field_a + field_b"
+
+  // Visibility
+  visibility: enum ["visible", "internal", "debug"]
+
+  // Execution
+  runTrigger: enum ["on_extraction", "on_demand", "scheduled"]
+  cacheDuration: number | null  // Seconds, null = no cache
+}
+```
+
+**Benefits:**
+
+- Standardized AI behavior patterns
+- Better extraction engine optimization
+- Clearer semantics for cross-form dependencies
+- Support for non-LLM calculations
+
+**Effort:** Medium | **Dependencies:** Derived fields in chat (A)
+
+---
+
+### 11.10 Explicit Pinned Field Reference
+
+**Concept:** Replace implicit "first field is pinned" with explicit foreign key reference.
+
+**Current Behavior:**
+
+- Collection groups implicitly treat first field (by order) as pinned
+- UX shows first field specially
+- No schema enforcement
+
+**Proposed Schema:**
+
+```typescript
+form_group {
+  // ... existing fields
+
+  pinnedFieldId: string (FK to form_field) | null
+  // Required when multi = true
+  // Must reference a field in this group
+}
+```
+
+**Validation Rules:**
+
+- If `multi = true`, `pinnedFieldId` is required
+- `pinnedFieldId` must reference a field where `formGroupId = this.id`
+- Cannot delete a field that is pinned
+- Changing pinned field requires explicit action
+
+**Migration:**
+
+```sql
+-- Backfill: set pinnedFieldId to first field by order
+UPDATE form_group g
+SET pinned_field_id = (
+  SELECT id FROM form_field f
+  WHERE f.form_group_id = g.id
+  ORDER BY f.order ASC
+  LIMIT 1
+)
+WHERE g.multi = true;
+```
+
+**Benefits:**
+
+- Explicit is better than implicit
+- Prevents accidental breakage when reordering fields
+- Cleaner API contracts
+
+**Effort:** Small | **Dependencies:** None (can be done independently)
+
+---
+
+### 11.11 Auto-Refactor Suggestions
+
+**Concept:** Use extraction analytics to proactively suggest form improvements.
+
+**Data Collection:**
+
+```typescript
+field_extraction_metrics {
+  id: string
+  formFieldId: string (FK)
+  formVersionId: string (FK)
+
+  // Aggregated metrics
+  extractionCount: number
+  successRate: number  // 0-1
+  averageConfidence: number
+  manualCorrectionRate: number
+
+  // Common issues
+  commonErrors: {
+    type: string  // "wrong_type", "missed", "hallucinated"
+    count: number
+    examples: string[]
+  }[]
+
+  // Time range
+  periodStart: timestamp
+  periodEnd: timestamp
+}
+```
+
+**Suggestion Engine:**
+
+```typescript
+interface RefactorSuggestion {
+  type: "rename" | "retype" | "add_hint" | "split_field" | "merge_fields"
+        | "make_collection" | "remove_field" | "add_derived"
+
+  targetFieldId: string
+
+  reason: string  // "This field has 45% correction rate"
+
+  suggestedChange: {
+    before: Partial<Field>
+    after: Partial<Field>
+  }
+
+  confidence: number
+
+  expectedImprovement: string  // "Estimated +20% accuracy"
+}
+```
+
+**UI:**
+
+- "Suggestions" panel in form editor
+- Notification: "3 forms have improvement suggestions"
+- One-click apply suggestion
+- Dismiss with feedback (improves future suggestions)
+
+**Effort:** Extra Large | **Dependencies:** Extraction analytics infrastructure, significant ML/heuristics work
+
+---
+
+### 11.12 Workflow as First-Class Entity
+
+**Concept:** Formalize workflows with ordered steps, each referencing specific form versions.
+
+**Current State:**
+
+- Workflows exist but forms are loosely associated
+- No explicit step ordering
+- No version pinning
+
+**Proposed Schema:**
+
+```typescript
+workflow {
+  id: string
+  organizationId: string (FK)
+  name: string
+  description: string
+  status: enum ["draft", "active", "archived"]
+  createdAt: timestamp
+  updatedAt: timestamp
+}
+
+workflow_step {
+  id: string
+  workflowId: string (FK)
+
+  order: number  // Step sequence
+  name: string   // "Upload Bank Statements"
+
+  formId: string (FK)
+  formVersionId: string (FK)  // Pinned to specific version
+
+  // Step behavior
+  isRequired: boolean
+  allowMultipleSubmissions: boolean
+
+  // Conditions (future)
+  skipCondition: string | null  // Expression: "previous_step.field_x == 'N/A'"
+}
+
+workflow_version {
+  id: string
+  workflowId: string (FK)
+  version: number
+  snapshot: jsonb  // Full workflow structure
+  status: enum ["draft", "active", "archived"]
+  createdAt: timestamp
+}
+```
+
+**Features:**
+
+- Visual workflow designer (drag-drop steps)
+- Version workflows independently of forms
+- Upgrade individual steps to new form versions
+- Workflow templates
+
+**Effort:** Extra Large | **Dependencies:** Version control (H), workflow builder mode (J)
+
+---
+
+## 12. Ideas Summary Table
+
+| ID | Idea | Effort | Priority Suggestion | Dependencies |
+|----|------|--------|---------------------|--------------|
+| 11.1 | Capability Model | M | P1 - prevents bad forms early | None |
+| 11.2 | Form Metadata Schema | S | P1 - enables templates & search | None |
+| 11.3 | Version Status Workflow | M | P2 - adds governance | H |
+| 11.4 | Role-Based Access Control | M | P2 - enterprise need | Org model |
+| 11.5 | Audit Logging | S-M | P2 - compliance | None |
+| 11.6 | Form Version Test Harness | L | P4 - high value, big build | H |
+| 11.7 | Change Impact Analysis | L | P4 - requires dependency graph | H, I |
+| 11.8 | Environment Separation | L | P3 - enterprise feature | H |
+| 11.9 | Enhanced DerivedFieldConfig | M | P2 - useful standardization | A |
+| 11.10 | Explicit Pinned Field Reference | S | P1 - cleaner model | None |
+| 11.11 | Auto-Refactor Suggestions | XL | Future - ML-heavy | Analytics infra |
+| 11.12 | Workflow as First-Class Entity | XL | Future - major rearchitecture | H, J |