ecoportal-api-graphql 1.3.5 → 1.3.9

data/.ai-assistance/code/diff_service_deep_dive.md

@@ -0,0 +1,192 @@
+# Code Spec: DiffService Deep Dive
+
+**Scope:** `DiffService`, `ClassicDiffService`, `HashDiffNesting`, `AsInput` — bugs discovered
+during 4.4 spec work, current state, and what still needs fixing.
+**Last updated:** 2026-06-05
+
+---
+
+## Current State (post-4.2 / pre-4.4 completion)
+
+### What works
+- **Flat diffs on simple models** (no cascade): `change_data` produces correct output
+  after key normalisation fix (see below).
+- **`patchVer` injection** in `AsInput#as_input`.
+- **`from_model` delegation** via `Logic::Input.from_model`.
+- **`ContractorEntity::Update.from_model`** IdDiff reshaping.
+
+### What is still broken / incomplete
+See TODO 4.4 for the spec gaps. Key issues:
+
+---
+
+## Bug 1 — String vs Symbol Keys in `change_data` (ROOT CAUSE OF MOST FAILURES)
+
+`ClassicDiffService#prev_doc` returns `model.original_doc` — a hash with **string keys**
+(set via `JSON.parse` in `DoubleModel#initialize`).
+
+`HashDiffNesting#change_data` converts each key to symbol via `key.to_sym` and then
+looks up the value in `b` (prev_doc) with:
+```ruby
+if b&.key?(key)   # key is now :name (symbol)
+  b_value = b[key]
+```
+
+Since `b` has `'name'` (string), `b.key?(:name)` is **always false**. Every field appears
+as "changed" (new value vs nil previous). The diff is wrong.
+
+### Fix needed in `DiffService#classic_diff`
+
+Normalise both docs to symbol keys before comparison:
+```ruby
+def classic_diff(flat: flat?, ignore: [])
+  curr = keys_to_sym_deep(curr_doc(flat: flat) || {})
+  prev = keys_to_sym_deep(prev_doc(flat: flat) || {})
+  
+  data = change_data(curr, prev, ignore: to_a(ignored, ignore))
+  return nil if data == NO_CHANGES
+  
+  id = get_id(curr, exception: false)
+  id ? { id: id }.merge(data) : data
+end
+```
+
+`keys_to_sym_deep` is available as an instance method via `include HashHelpers`.
+
+---
+
+## Bug 2 — `dig_path?` Returns True for Any Single-Element Path
+
+The `DoubleModel::HashHelpers` implementation:
+```ruby
+def dig_path?(obj, keys)
+  return false unless obj.respond_to?(:[])
+  return true  if keys.length == 1   # ← BUG: doesn't check key existence
+  dig_path?(obj[keys.first], keys[1..])
+end
+```
+
+This always returns `true` for single-element key paths regardless of whether the key
+actually exists. For `diff_reduce`, this causes:
+- `dig_path?({id:, name:}, ['actionCategory'])` → **true** (wrong! actionCategory not present)
+- `dig_set!(result, ['actionCategory'], nil)` adds spurious nil entries to the diff
+
+**Fix in `HashDiffNesting#dig_path?`:**
+```ruby
+def dig_path?(obj, keys)
+  return false unless obj.is_a?(Hash)
+  return obj.key?(keys.first) if keys.length == 1   # check actual existence
+  return false unless obj.key?(keys.first)
+  dig_path?(obj[keys.first], keys[1..])
+end
+```
+
+---
+
+## Bug 3 — `diff_reduce` Cannot Add New Cascade Keys
+
+`diff_reduce` was designed to REPLACE existing cascade keys in the flat diff. But
+cascade attributes are always STRIPPED from the flat diff (`doc_with_non_cascaded_attributes`).
+So the keys are never in the flat diff, and `diff_reduce` never adds them.
+
+### Current broken flow for nested change only:
+1. `dueDate` changed on an Action, nothing else changed
+2. `classic_diff(flat: true)` → `nil` (dueDate stripped, no flat-level changes)
+3. `diff_reduce(nil)` → all `dig_path?(nil, ...)` return `false` → cascade never processed
+4. Result: `nil` — the nested change is LOST
+
+### Current broken flow for mixed change:
+1. `name` changed AND `dueDate` changed
+2. `classic_diff(flat: true)` → `{id:, name: 'Updated'}` (flat change only)
+3. `diff_reduce({id:, name: 'Updated'})`:
+   - `dig_path?({id:, name:}, ['dueDate'])` → `false` (dueDate not in flat diff) → skip
+4. Result: `{id:, name: 'Updated'}` — dueDate change LOST
+
+### Fix needed in `diff_reduce`:
+
+Remove the `dig_path?` gate. Instead, ADD cascade diffs directly:
+```ruby
+def diff_reduce(init, ignore: [])
+  subject.cascaded_reduce(init, recurs: false) do |result, obj, _key, key_path, _trace|
+    next result if key_path.empty?
+    next result if obj == subject
+    next result if root?(obj)
+    next result unless obj.is_a?(Ecoportal::API::Common::GraphQL::Model)
+    
+    value = obj.as_update(ignore: ignore)
+    next result if value.nil?
+    
+    result ||= {}
+    dig_set!(result, key_path, value)
+    result
+  end
+end
+```
+
+Additionally, `diff` must ensure `id:` is present even when init was nil:
+```ruby
+def diff(flat: flat?, ignore: [])
+  flat_result = classic_diff(flat: true, ignore: ignore)
+  return flat_result if flat
+  
+  result = diff_reduce(flat_result, ignore: ignore)
+  return nil if result.nil?
+  
+  # Guarantee id: is in result when cascade added something
+  result[:id] ||= get_id(keys_to_sym_deep(curr_doc(flat: true) || {}), exception: false)
+  result
+end
+```
+
+---
+
+## Bug 4 — `ecoportal-api-v2` `_cascaded_attributes_trace` (FIXED in 6a2b1b5)
+
+Block params were `|out, (attribute, obj_k)|` (swapped) — `attribute` received the
+trace hash instead of the attribute symbol, causing `TypeError: Hash is not a symbol`.
+Fixed to `|(attribute, obj_k), out|`. Requires rebuild of ecoportal-api-v2 gem.
+
+---
+
+## Missing Methods in `DiffService` (FIXED in 0b68a6b)
+
+`diff_reduce` calls `dig_path?`, `dig_delete!`, `dig_set!` which are defined in
+`DoubleModel::HashHelpers` (ecoportal-api-v2) but NOT included in `DiffService`.
+
+Added these three methods to `HashDiffNesting::InstanceMethods`. See
+`lib/ecoportal/api/common/graphql/model/diffable/hash_diff_nesting.rb`.
+
+---
+
+## `diff_reduce` — ArrayModel Safety (FIXED in 0b68a6b)
+
+`passarray :fieldName` creates `ArrayModel` subclasses inheriting from `DoubleModel`.
+These respond to `as_update` but use the ecoportal-api-v2 signature (no kwargs).
+
+Added guard: `next result unless obj.is_a?(Ecoportal::API::Common::GraphQL::Model)`.
+This limits cascade-diffing to GraphQL model objects only.
+
+---
+
+## Recommended Implementation Order for 4.4 Completion
+
+1. Fix `DiffService#classic_diff` key normalisation (Bug 1 — most impactful)
+2. Fix `HashDiffNesting#dig_path?` to check key existence (Bug 2)
+3. Redesign `diff_reduce` to ADD rather than REPLACE cascade keys (Bug 3)
+4. Update `diff` to ensure `id:` presence and proper no-change return
+5. Update specs to pass for all scenarios
+
+---
+
+## File Locations
+
+| File | Role |
+|------|------|
+| `lib/.../model/diffable/diff_service.rb` | Main DiffService — `classic_diff`, `diff`, `diff_reduce` |
+| `lib/.../model/diffable/classic_diff_service.rb` | Base: `curr_doc`, `prev_doc`, `change_diff` dispatch |
+| `lib/.../model/diffable/hash_diff_nesting.rb` | `change_data`, `patch_*`, `dig_*` helpers |
+| `lib/.../model/as_input.rb` | `as_input` instance + class methods, `patchVer` injection |
+| `lib/.../graphql/logic/input.rb` | `Logic::Input.from_model` base |
+| `lib/.../input/contractor_entity/update.rb` | `from_model` override with IdDiff reshaping |
+| `spec/.../model/diff_service_spec.rb` | DiffService specs (WIP — partial failures) |
+| `spec/.../model/as_input_spec.rb` | AsInput + from_model specs (WIP) |

data/.ai-assistance/code/ecoPortal_architecture/00_overview_and_index.md

@@ -0,0 +1,55 @@
+# EcoPortal Architecture — Agent Reference Manual
+
+**Purpose:** Enable an AI agent to read this folder, then immediately navigate the
+ecoPortal source code, customer documentation, and integration gems with full context.
+Covers: platform purpose, data model, naming conventions, API layers, workflows,
+search, integrations, and documentation gaps.
+
+**Session:** Accumulated 2026-06-07 from developer context, production Insomnia queries,
+corpus documents, eco-helpers source audit, and GraphQL schema introspection.
+
+---
+
+## Document Index
+
+| File | Content |
+|------|---------|
+| `00_overview_and_index.md` | This file — platform summary + index |
+| `01_terminology_dictionary.md` | Every name things are called — Rosetta Stone for disambiguating |
+| `02_data_model.md` | Page hierarchy, types, MongoDB structure, genome, field ID |
+| `03_api_layers.md` | APIv2 vs GraphQL — differences, retirement timeline, access |
+| `04_graphql_queries_mutations.md` | All known queries/mutations, fragments, CommonPageUnion |
+| `05_page_workflows.md` | Create/Update/Archive/Submit — 2-step sequences, stage logic |
+| `06_search_and_filters.md` | Org search vs register search, SearchConf, filter operations |
+| `07_data_fields.md` | All 20+ field types, DataFieldInput, update/addition patterns |
+| `08_stages_sections.md` | Stage lifecycle, permissions, sections, HasSectionsInterface |
+| `09_people_contractors_locations.md` | PersonMember, ContractorEntity, IdDiff, location tags |
+| `10_forces_workflow_builder.md` | Forces (AngularJS legacy), Workflow Builder (in progress) |
+| `11_integration_gems.md` | eco-helpers, ecoportal-api-v2, ecoportal-api-graphql chain |
+| `12_ai_documentation_sources_gaps.md` | All knowledge sources + gaps + improvements for AI agents |
+
+---
+
+## Platform Summary
+
+EcoPortal is a SaaS **governance, risk and compliance (GRC)** platform built for
+enterprise customers. It enables organisations to manage:
+- Risk registers (pages = records in a workflow-driven register)
+- Incident reporting
+- Contractor management
+- Action tracking
+- Location/reporting structure management
+- Visitor management (check-in)
+- Notifications, dashboards, analytics
+
+**Technology stack:**
+- Backend: Ruby on Rails, MongoDB (NoSQL document store)
+- Frontend: AngularJS (legacy, being replaced) → React (current, in progress)
+- APIs: APIv2 REST (being retired), GraphQL (current standard)
+- Search/analytics: Elasticsearch
+- Deployment: Docker, cloud-hosted
+
+**Key architectural principle:** MongoDB's flexible document model means page records
+("instances") are cloned from templates. Field structure is defined in the template;
+field data is stored per-instance. Field IDs are assigned by MongoDB on creation and
+are instance-specific (not predictable from the template alone).

data/.ai-assistance/code/ecoPortal_architecture/01_terminology_dictionary.md

@@ -0,0 +1,181 @@
+# EcoPortal Terminology Dictionary
+
+An AI agent MUST read this before working with ecoPortal code or customer requests.
+The same concept has different names depending on who is talking and which layer of
+the system is being discussed. Misunderstanding these leads to wrong assumptions.
+
+---
+
+## Core Entity Names
+
+### Pages / Records / Ooze / Forms
+
+| Context | Name used | Notes |
+|---------|-----------|-------|
+| Customer-facing (product UI) | **Page**, **Record** | "Page" is the UI term; "Record" used in registers |
+| eco-helpers scripts (internal) | **Ooze** | Internal dev shorthand, from legacy gem name |
+| eco-helpers class names | `OozeBase`, `RegisterUpdate` | Script use case base classes |
+| Confluence docs | **Page** | |
+| APIv2 gem | `Ecoportal::API::V2::Page` | |
+| GraphQL schema | `BasicPage`, `PhasedPage`, `PageUnion` | Two concrete types |
+| GraphQL queries | `page(id:)`, `pages(...)` | |
+| ecoportal-api-graphql gem | `Model::Page::Basic`, `Model::Page::Phased` | |
+
+**Disambiguation rule:** When a customer says "record" or "page", they mean a page
+instance — NOT a template. Templates are a separate concept (see below).
+
+### Stages / Workflow Steps / Flow Nodes
+
+| Context | Name used | Notes |
+|---------|-----------|-------|
+| Customer-facing (product UI) | **Stage** | |
+| Backend Enzyme namespace (old) | **flow_nodes** | Old Rails model name |
+| Backend NewEp namespace (current) | **stages** | |
+| APIv2 | **stages** | |
+| GraphQL schema | `Stage`, `StageIndex`, `StageStateEnum` | |
+| eco-helpers | **stages** | `page.stages[name]`, `page.stages.ordered` |
+
+### Sections / Content Containers / Flow Node Sub-elements
+
+| Context | Name used | Notes |
+|---------|-----------|-------|
+| Customer-facing (UI) | **Section** | Within a stage |
+| Backend Enzyme namespace (old) | **flow_nodes** sub-elements | No clean separation |
+| Backend NewEp namespace (current) | **sections** | `HasSectionsInterface` |
+| APIv2 | **sections** | `page.sections` |
+| GraphQL schema | `SectionUnion`, `ContentSection`, `SplitSection` | Two section types |
+| eco-helpers | **sections** | `entry.sections.get_by_type(:content_section)` |
+
+**Important:** BasicPage (no stages) has sections DIRECTLY. PhasedPage has sections
+WITHIN each stage. This is why `page.sections` on a PhasedPage requires knowing which
+stage you're in.
+
+### Data Fields / Components / Membranes / Field Instances
+
+| Context | Name used | Notes |
+|---------|-----------|-------|
+| Customer-facing (UI) | **Field** | Form field within a section |
+| Backend Enzyme namespace (old) | **Membrane**, `Enzyme::Membrane` | Old model name |
+| Backend NewEp namespace (current) | **DataField**, `NewEp::DataField` | |
+| APIv2 | **component** | `page.components`, `component.type` |
+| GraphQL schema | `DataFieldUnion`, `DataFieldsInterface` | 20+ concrete types |
+| eco-helpers | **components** | `entry.components.get_by_type(:plain_text)` |
+| ecoportal-api-graphql gem | `DataField::PlainText`, `DataField::Select`, etc. | |
+
+**CRITICAL:** A "field" in the UI is an INSTANCE of a field definition. The template
+defines the field structure (label, type, options). The page instance has the actual
+field with server-assigned MongoDB ID and stored value. The instance field ID CANNOT be
+predicted from the template alone — must be fetched after creation via `buildFromTemplate`.
+
+### Registers / Registries / Page Collections
+
+| Context | Name used | Notes |
+|---------|-----------|-------|
+| Customer-facing (UI) | **Register** | A themed collection of pages with a shared template |
+| GraphQL schema | `Register`, `register(id:)` | |
+| eco-helpers | **register** | `registers.search(register_id, ...)` |
+
+### Templates
+
+| Context | Name used | Notes |
+|---------|-----------|-------|
+| Customer-facing (UI) | **Template** | A "page type" defining field structure |
+| GraphQL schema | `PageUnion` (templates are also pages), `Template` type | |
+| eco-helpers | **template** | `pages.get_new(template_id)` |
+
+**IMPORTANT:** Engineering does NOT want templates updated via GraphQL currently.
+Template editing uses the old AngularJS front-end. Template mutations exist in the
+schema but are not production-approved for scripting.
+
+---
+
+## Field-Level Terminology
+
+### ref / Field ID / ES Key / System ID (future)
+
+The `ref` property on a data field (`Enzyme::Membrane` / `NewEp::DataField`) is the
+**Elasticsearch index key** for that field. It determines how analytics, charts, filters,
+and cross-register configurations identify the field.
+
+**Composition:** `type_shorthand.hash_of_label`
+- `type_shorthand`: e.g. `plain_text`, `date`, `select_str`, `select_num`, `gauge`,
+  `rich_text`, `actions_list`, `files`
+- Hash of the label name (truncated when label > 3 characters)
+- Formula: `Ecoportal::API::Common::Content::StringDigest` in `ecoportal-api-v2`
+
+**Future:** The team is planning to formalise this as a user-visible "Field ID" (as
+opposed to the internal MongoDB `id`). Currently it is only used internally.
+
+**In search/sorters:** The `key` field in sorters (`{ key: 'updated_at', direction: 'asc' }`)
+references this ref value, not the MongoDB field `id`.
+
+### patchVer / Optimistic Concurrency
+
+`patchVer: Int!` on every page. The server uses this for optimistic concurrency control.
+**Any update mutation MUST include the current `patchVer` value fetched from the page.**
+If two scripts try to update the same page concurrently, the second will get a stale-patchVer
+error. Scripts must re-fetch the page to get the current `patchVer` before retrying.
+
+### Genome Signature (mostly unused)
+
+The genome signature was designed to pair fields ACROSS pages and their source templates.
+Purpose: know that "this field in page X is the same field as this field in template Y".
+**Current state:** Mostly unused. The Migrator Rails service that used it was abandoned.
+Not indexed in ES. Many fields have broken genomes due to bulk tech scripts that added
+fields without copying from the template. Do NOT design scripting around genome.
+
+### Forces (legacy front-end snippets)
+
+Forces are AngularJS embedded code snippets that attach to specific stages via bindings.
+They implement conditional field behaviour (show/hide sections, computed risk matrices).
+**Being replaced** by the Workflow Builder. Not accessible or useful via GraphQL.
+Low scripting priority.
+
+---
+
+## State / Status Names
+
+### Page State (`StateEnum`)
+
+| Value | Meaning |
+|-------|---------|
+| `active` | Normal working state |
+| `archived` | Soft-deleted/hidden — can be unarchived |
+| `draft` | Not yet published (also `draft: Boolean` field) |
+| `inprogress` | In a workflow stage (used in some contexts) |
+
+### Stage State (`StageStateEnum`)
+
+| Value | Meaning |
+|-------|---------|
+| `pending` | Not yet started |
+| `inprogress` | Currently active |
+| `complete` | Completed (all tasks done) |
+
+### Task State
+
+Tasks (`TaskInterface`, `CompletePage`, `ReviewPage`) have their own states related to
+completion. Stage close-out is triggered by task completion.
+
+---
+
+## API Layer Names
+
+| Layer | Name | Status |
+|-------|------|--------|
+| Legacy REST | **APIv2**, `ecoportal-api-v2` gem | Being retired (3 weeks from 2026-06-07) |
+| Current GraphQL | **GraphQL API**, `ecoportal-api-graphql` gem | Current standard |
+| Internal REST (auth) | **APIv1** (OAuth) | Still in use for authentication only |
+| Future | **APIv3** (page render / custom exports) | In development, not production |
+
+---
+
+## Backend Namespace Names
+
+| Namespace | Description | Location |
+|-----------|-------------|----------|
+| `Enzyme` | Old Rails model namespace (AngularJS era) | `C:\docker\ecoPortal_master\` |
+| `NewEp` | Current Rails model namespace (React era migration) | Same repo |
+| `Ecoportal::API::V2` | REST API gem | `C:\ruby_scripts\git\ecoportal-api-v2` |
+| `Ecoportal::API::GraphQL` | GraphQL gem | THIS REPO |
+| `Eco` (eco-helpers) | Integration scripts helper gem | `C:\ruby_scripts\git\eco-helpers` |

data/.ai-assistance/code/ecoPortal_architecture/02_data_model.md

@@ -0,0 +1,192 @@
+# EcoPortal Data Model
+
+---
+
+## Top-Level Hierarchy
+
+```
+Organisation
+  ├── Registers (themed page collections)
+  │     ├── Templates (page type definitions)
+  │     │     ├── Stages (workflow steps — PhasedPage templates)
+  │     │     │     └── Sections
+  │     │     │           └── DataField definitions (with label, type, options)
+  │     │     └── Sections (for BasicPage templates)
+  │     │           └── DataField definitions
+  │     └── Pages / Instances (created from templates)
+  │           ├── [BasicPage] Sections → DataFields (actual data, server IDs)
+  │           └── [PhasedPage] Stages → Sections → DataFields (actual data, server IDs)
+  ├── People (PersonMember)
+  ├── Contractor Entities
+  ├── Location Structure (reporting structure / RS)
+  │     └── Location Nodes (hierarchical tree)
+  ├── Actions (standalone or linked to pages)
+  └── User Groups / Permissions
+```
+
+---
+
+## `externalId` — per-org unique key for integrations (VERIFIED, Oscar 2026-07-02)
+
+`externalId` exists on models (pages, and other objects) as the **integration pairing key**:
+
+- **Uniqueness is per-organization, per-model:** an `externalId` must be unique among objects of
+  the *same model* within one ecoPortal organization. It is the mechanism the platform uses to
+  **prevent double-ups** and to give a customer integration a stable way to **pair its own record
+  ids with ecoPortal's objects** (set `externalId` = the customer's key; look up / upsert by it).
+- **Implication for integrations:** when you control the source data, prefer keying create/upsert
+  on `externalId` (guaranteed unique, server-enforced) over searching a data-field value. A
+  field-value search can miss (wrong key, analyzer semantics — see the cans-upsert dup incident in
+  `search_filters.md`) and silently create duplicates; an `externalId` collision is rejected.
+- **Archiving contingency:** clear the `externalId` only when archiving is a *cleanup of a failed /
+  duplicate creation* (so the freed id can be reused); a genuine historical archive should **keep**
+  its `externalId`. The backend `archivePage` mutation takes only `id` (no clear-externalId arg), so
+  this is a **client composition**: `updatePage(page:{externalId:null})` + `archivePage`, in one
+  document (the maintainer's Insomnia sample gates the updatePage with `@include(if:$blank_external_id)`).
+  WIRED as **`Builder::Page#archive(clear_external_id: false)`** (default keeps it) — implemented as
+  two serial mutations (blank-then-archive) reusing the existing update/archive mutations.
+- **Note:** the cans-upsert case does NOT use `externalId` (it keys on the `Identifier:` data
+  field) — which is exactly why a wrong search key produced duplicates. An `externalId`-based
+  upsert would have been collision-safe.
+
+---
+
+## Page Types (PageUnion)
+
+The GraphQL `PageUnion = BasicPage | PhasedPage`. Both implement `BasePageInterface`.
+
+### BasicPage
+
+- Has sections DIRECTLY (no stages)
+- Used for: simple forms, non-workflow records
+- `implements: BasePageInterface, HasSectionsInterface`
+- `sections: [SectionUnion!]!` — direct access
+
+### PhasedPage
+
+- Has stages, each stage has sections
+- Used for: workflow-driven records (risk assessments, incident reports, inspections)
+- `implements: BasePageInterface`
+- `stages: [Stage!]!` — access via StageCollection
+- `stagesIndex: [StageIndex!]!` — lightweight stage summary
+- `currentStage: Stage!` — the currently active stage
+
+### Identifying the Type
+
+Always include `__typename` in page queries. The server returns `'BasicPage'` or
+`'PhasedPage'`. In the ecoportal-api-graphql gem, `Model::PageUnion.new(doc)` dispatches
+on `__typename` to create the right concrete class.
+
+---
+
+## MongoDB Document Structure
+
+Pages are stored as MongoDB documents. The document structure is flexible — field
+instances are embedded within the stage/section hierarchy.
+
+**Key implications:**
+1. **Field IDs are MongoDB ObjectIds** assigned at creation. They are DIFFERENT from
+   the template's field IDs. An instance field ID cannot be predicted — must be fetched.
+2. **patchVer** is an integer version counter incremented on every save. Required for
+   all update mutations.
+3. **Partial updates** — the GraphQL `updatePage` mutation only changes the fields you
+   specify. The rest remain unchanged. This is different from APIv2 which sometimes sent
+   the full document.
+4. **Stages are embedded** within the page document. Fetching a page gets all stages.
+
+---
+
+## Template vs Instance
+
+| Aspect | Template | Instance (Page) |
+|--------|----------|-----------------|
+| Created by | Admins via UI | Scripts or end users |
+| Field IDs | Template-specific ObjectIds | NEW ObjectIds assigned on creation |
+| patchVer | Yes | Yes |
+| Stage structure | Defines stage names/order | Inherited at creation, runtime state |
+| DataField content | Default values only | Actual user-entered values |
+| GraphQL access | `templates(...)` query | `pages(...)` or `page(id:)` |
+
+**CRITICAL for scripting:** After `buildFromTemplate`, the response contains the
+INSTANCE field IDs. These are the IDs to use for `dataFields.updates` in
+`createFromTemplate`. The template field IDs are irrelevant for instance operations.
+
+---
+
+## Stage Model (PhasedPage)
+
+Each stage in a PhasedPage:
+- Has a unique `id` (MongoDB ObjectId)
+- Has `name` (the stage name configured in the template)
+- Has `ordering` (display order, integer)
+- Has `state` (`StageStateEnum`: pending/inprogress/complete)
+- Has `active: Boolean` (whether stage is currently active for input)
+- Has `started: Boolean` (whether any input has been recorded)
+- Has `sections: [SectionUnion!]!` — the stage's content sections
+- Has task references: `latestCompletePageTask`, `latestReviewPageTask`, `activeTask`
+
+**Stage permissions:**
+- Each stage can have user group restrictions
+- People fields can grant direct stage access based on their values
+- Configuration enum on people fields determines scope (stage-specific vs page-wide)
+- GraphQL silently omits stages the user cannot access (no access errors)
+
+**Stage transitions:**
+- Triggered by task completion (complete task = stage done → next stage starts)
+- Two task types: `CompletePage` (fill-in) and `ReviewPage` (sign-off)
+- Submitting a stage via GraphQL `updatePage(stageId:, submit: true)` triggers the
+  server-side task assignment rules (who gets the next task per workflow config)
+
+---
+
+## Section Model
+
+Two types (`SectionUnion = ContentSection | SplitSection`):
+
+### ContentSection
+- `heading: String` — section heading
+- `dataFields: [DataFieldUnion!]!` — fields in this section
+
+### SplitSection
+- `heading: String`
+- `leftDataFields: [DataFieldUnion!]!`
+- `rightDataFields: [DataFieldUnion!]!`
+
+Both implement `HasSectionsInterface`. When a section's fields need to be accessed
+together, merge `dataFields + leftDataFields + rightDataFields`.
+
+---
+
+## Genome Signature (Field Pairing — Mostly Unused)
+
+Each field definition (in template and in instances) carries a **genome signature** —
+a hash intended to identify that a field is "the same" across template and instances.
+
+**Original design:** Enable the Migrator service to propagate template changes to
+existing pages by pairing fields via genome.
+
+**Current reality:**
+- Largely abandoned — the Migrator service is inactive
+- Many instances have broken genomes (bulk tech scripts that added fields
+  without copying from the template gave them random creation-time genomes)
+- Not indexed in Elasticsearch
+- Unknown whether `NewEp` fully ported the genome from `Enzyme`
+- Do NOT design scripting logic around genome — it cannot be relied upon
+
+---
+
+## Location Structure (Reporting Structure)
+
+The organisation has a hierarchical location tree. Locations are used for:
+- Tagging pages (assigning pages to locations/departments)
+- Filtering in searches (`location_ids` filter)
+- Permissions scoping
+- Analytics/dashboard scoping
+
+The location tree is managed via dedicated mutations (`draft` system for structural edits).
+Location nodes have: `id`, `name`, `weight`, `archived`, `archivedToken`, `classifications`,
+`ancestors`, `parent`, `children`.
+
+**`baseTags` vs `otherTags`:**
+- `baseTags` — location tags inherited from the base template configuration
+- `otherTags` — additional free-text tags attached to a page instance