@hippo-digital/hippocampus 1.0.0-rc.1

package/docs/hippocampus-technical.md

@@ -0,0 +1,373 @@
+# Hippocampus Technical Model
+
+Hippocampus is a structured design knowledge store for NHS Prototype Kit projects.
+
+It is not a graph database. It is a set of validated JSON source files, helper functions and Nunjucks views that turn linked records into browsable knowledge views under `/hippocampus`.
+
+## Source Layout
+
+Canonical source data lives in:
+
+```text
+hippocampus/source/
+```
+
+The current source collections are:
+
+```text
+project.json
+project-context.json
+users.json
+needs.json
+scenarios.json
+journeys.json
+screens.json
+evidence.json
+decisions.json
+assumptions.json
+research-rounds.json
+participants.json
+insights.json
+design-actions.json
+relationships.json
+```
+
+Supporting material lives near the source:
+
+```text
+hippocampus/inbox/research/        new research artefacts to import
+hippocampus/inbox/project/         new project overview artefacts to import
+hippocampus/imports/research/      reviewable import drafts
+hippocampus/imports/project/       reviewable project context drafts
+hippocampus/manifests/             import and promotion ledgers
+hippocampus/source-artefacts/      copied evidence files
+```
+
+Generated outputs belong in:
+
+```text
+artefacts/
+```
+
+Deleting `artefacts/` should not break validation or rendering.
+
+## Runtime Flow
+
+The app loads source data through:
+
+```text
+node_modules/@hippo-digital/hippocampus/lib/load-knowledge.js
+```
+
+The schema lives in:
+
+```text
+node_modules/@hippo-digital/hippocampus/lib/schema.js
+```
+
+The runtime flow is:
+
+```text
+read JSON files
+  -> validate with Zod schemas
+  -> build ID indexes
+  -> build entity index
+  -> build view models
+  -> render Nunjucks views
+```
+
+The main public route surface is:
+
+```text
+/hippocampus
+```
+
+Project context is rendered at:
+
+```text
+/hippocampus/project
+```
+
+Legacy `/knowledgeable` URLs redirect to `/hippocampus`.
+
+## ID Indexes
+
+Every source record has a stable lowercase kebab-case `id`.
+
+At runtime, `buildIndex()` creates lookup maps:
+
+```text
+knowledge.index.users
+knowledge.index.needs
+knowledge.index.scenarios
+knowledge.index.journeys
+knowledge.index.screens
+knowledge.index.evidence
+knowledge.index.decisions
+knowledge.index.assumptions
+knowledge.index.researchRounds
+knowledge.index.participants
+knowledge.index.insights
+knowledge.index.designActions
+knowledge.index.relationships
+```
+
+This lets views and graph functions resolve references quickly without repeatedly scanning whole arrays.
+
+## Entity Index
+
+`buildEntityIndex()` creates one cross-collection entity map:
+
+```text
+knowledge.index.entities
+```
+
+It normalises different record types into a common shape:
+
+```json
+{
+  "id": "need-example",
+  "type": "Need",
+  "label": "A human-readable label",
+  "collection": "needs",
+  "item": {}
+}
+```
+
+This index is what allows `relationships.json` to link across different kinds of records.
+
+## Project Context
+
+`project-context.json` stores generic project overview material outside the relationship graph.
+
+It is intended for:
+
+- purpose and vision
+- problem statement
+- service proposition
+- project-level users
+- channels
+- policies and regulations
+- constraints
+- risks
+- milestones
+- source facts
+- source overview artefacts
+
+The import flow mirrors research import:
+
+```text
+hippocampus/inbox/project/
+  -> npx hippocampus import project-context
+  -> hippocampus/imports/project/*.json
+  -> npx hippocampus promote project-context
+  -> hippocampus/source/project-context.json
+```
+
+It remains separate from research because overview boards usually mix facts, strategy, constraints and policy context rather than direct research findings.
+
+## Two Graph Models
+
+Hippocampus currently has two graph mechanisms.
+
+### Implied Graph
+
+Some relationships are embedded directly in source records.
+
+Examples:
+
+```text
+need.userId
+need.evidenceIds
+scenario.userId
+scenario.needIds
+journey.scenarioId
+journey.steps[].needIds
+journey.steps[].screenIds
+journey.steps[].evidenceIds
+journey.steps[].decisionIds
+screen.needIds
+```
+
+These produce an implied graph:
+
+```text
+user
+  -> needs
+      -> evidence
+  -> scenarios
+      -> needs
+      -> journeys
+          -> steps
+              -> screens
+              -> evidence
+              -> decisions
+```
+
+`getUserKnowledgeGraph(userId)` resolves this implied graph for one user. It powers pages such as:
+
+```text
+/hippocampus/users/policy-administrator
+```
+
+### Explicit Graph
+
+`relationships.json` is the explicit graph layer.
+
+Each relationship has:
+
+```json
+{
+  "id": "rel-example",
+  "from": "source-entity-id",
+  "type": "relationship-type",
+  "to": "target-entity-id"
+}
+```
+
+Allowed relationship types currently include:
+
+```text
+has-scenario
+has-need
+tests-need
+supported-by
+drives-journey
+renders-screen
+addresses-need
+uses-evidence
+records-decision
+has-participant
+produced-insight
+contributed-to
+supports-insight
+responds-to-insight
+resolves-action
+leads-to-action (legacy, no longer emitted)
+```
+
+Validation checks that every `from` and `to` ID points to a known entity.
+
+`getGraphExplorer()` reads `relationships.json`, enriches each relationship with labels and entity types, and powers:
+
+```text
+/hippocampus/graph
+```
+
+## Trace Tree
+
+The trace tree route is:
+
+```text
+/hippocampus/trace
+```
+
+`getTraceExplorer(entityId)` uses the same enriched explicit relationships as the graph explorer.
+
+For the selected entity, it builds:
+
+- `upstream`: incoming relationships where `relationship.to` is the current entity
+- `downstream`: outgoing relationships where `relationship.from` is the current entity
+
+Each branch is expanded recursively to a maximum depth with cycle protection. This lets the viewer represent navigable paths such as:
+
+```text
+old design -> research -> insight -> action -> new design -> content
+```
+
+The trace tree is generic. It does not hard-code those labels; it renders whatever entities and relationship types exist in `relationships.json`.
+
+## Evidence Impact
+
+Evidence impact uses the explicit graph to show what depends on a piece of evidence.
+
+Route:
+
+```text
+/hippocampus/evidence/:evidenceId/impact
+```
+
+The process is:
+
+```text
+start with evidence ID
+  -> find relationships where relationship.to is the current ID
+  -> add relationship.from as an impacted dependent
+  -> repeat from each dependent
+```
+
+So if the graph says:
+
+```text
+need -> supported-by -> evidence
+screen -> addresses-need -> need
+journey -> uses-evidence -> evidence
+```
+
+then the impact view can show which needs, screens, journeys and decisions may be affected if that evidence changes.
+
+Direction matters. Relationships should be modelled so that dependent things point to the thing they depend on.
+
+## Research Import
+
+The default import command is:
+
+```bash
+npx hippocampus research sync
+```
+
+This command:
+
+- scans `hippocampus/inbox/research`
+- fingerprints supported files
+- skips already-known artefacts
+- creates reviewable drafts in `hippocampus/imports/research`
+- updates `hippocampus/manifests/research.json`
+- asks before promoting drafts into canonical source data
+
+Research imports can create:
+
+```text
+research rounds
+participants
+insights
+design actions
+relationships
+```
+
+Promotion is deliberate. Imported draft data should be reviewed before it becomes canonical source.
+
+## Validation
+
+Run:
+
+```bash
+npx hippocampus validate
+```
+
+Validation checks:
+
+- source JSON shape
+- required fields
+- lowercase kebab-case IDs
+- enum values
+- route format
+- duplicate IDs
+- cross-record references
+- explicit relationship endpoints
+
+Run:
+
+```bash
+npx hippocampus doctor
+```
+
+This starts the Prototype Kit and checks that generated Hippocampus routes and linked prototype screens return HTTP 200.
+
+## Current Limits
+
+- The graph is rendered as cards and relationship lists, not an interactive network diagram.
+- The explicit graph is manually maintained or imported; it is not fully inferred from every source record.
+- Relationship direction needs consistency for impact analysis to work well.
+- There is no graph query language yet.
+- There is no schema migration system yet.

package/docs/prompt-architecture-guardrails.md

@@ -0,0 +1,39 @@
+# Prompt Architecture Guardrails
+
+The prompt pack is for repeatable source-model and prototype maintenance. Prompts should make agents safer, not more verbose.
+
+## Principles
+
+- Source data beats hard-coded pages.
+- Evidence beats assertion.
+- Assumption beats invented evidence.
+- Small linked records beat large narrative blobs.
+- Generated artefacts belong in `artefacts/`, not in the source model.
+
+## Prompt Shape
+
+Each prompt should include:
+
+- files to read first
+- source records that may be edited
+- relationship constraints
+- validation command
+- expected finish summary
+
+## Do Not
+
+- Ask agents to create real patient data.
+- Ask agents to bypass validation.
+- Ask agents to copy large external component references into this repo.
+- Ask agents to rewrite unrelated source records for style only.
+- Ask agents to treat generated artefacts as source of truth.
+
+## Useful Prompt Stages
+
+1. Generate or revise a service slice.
+2. Add user needs.
+3. Add scenarios.
+4. Create journeys and blueprint steps.
+5. Add prototype screens.
+6. Audit traceability, evidence and assumptions.
+7. Produce review artefacts and delivery summaries.

package/docs/repeatable-research-round-ingestion.md

@@ -0,0 +1,239 @@
+# Repeatable Research Round Ingestion
+
+This playbook describes how a team can turn research artefacts into Hippocampus source data without relying on a single opaque AI step.
+
+This is the canonical staged ingestion workflow. If another repo doc overlaps with the step-by-step process, this document wins.
+
+The pattern is:
+
+```text
+triage the folder            (optional, for large corpora)
+  -> drop artefact
+  -> script extracts text
+  -> AI structures the draft from extracted text only
+  -> human reviews evidence and gaps
+  -> promotion script validates and merges
+```
+
+Use this for every research round, show-and-tell deck, interview synthesis, usability test report or project overview artefact that should become part of the knowledge graph.
+
+## Principles
+
+- The script is responsible for deterministic extraction, fingerprints and repeatable file handling.
+- Detection is scoped to section headings. A slide or Word heading such as "Insights",
+  "Findings", "Design actions" or "Participants" makes the lines beneath it
+  candidates; generic headings yield nothing. `docs/research-source-file-conventions.md`
+  explains how to author artefacts so this works.
+- AI is responsible for structuring, synthesis and spotting links that simple text rules miss.
+- Humans are responsible for deciding whether the draft is true enough to promote.
+- Promotion scripts are responsible for schema validation and controlled writes to `hippocampus/source`.
+- A draft passing schema validation does not mean the content is good.
+
+## A Round Is Fieldwork, Not A File
+
+A research round can span many artefacts: a discussion guide, twelve scenario maps,
+a playback deck, an insight tracker. They all belong to one round.
+
+`research:sync` reads the round from the folder name, so
+`Round 16 - Patient Search in FHIR/03 Analysing the research/P27 ...docx` imports
+into `round-16` rather than inventing a round called `p27-detailed-scenario-map`.
+Pass `--round-id` to override it. Each artefact still produces its own draft, and
+insight and design-action IDs are namespaced per artefact so drafts from one round
+do not collide. Participants stay round-scoped, because the same participant code
+in two artefacts is one person.
+
+Promotion is idempotent for the records a round shares. Promoting a second artefact
+from the same round reuses the existing round record and merges participants rather
+than failing on duplicate IDs; it still fails if the round exists with a different
+title or date, which means two rounds have been given the same ID.
+
+Dates come from a `YYYY-MM-DD`, `YYYYMMDD` or `YYYY-MM` in the file or folder name.
+Where none is found the draft carries today's date and a warning saying so, because
+file modification dates do not survive a copy off a shared drive.
+
+## Folder Contract
+
+Research artefacts go in:
+
+```text
+hippocampus/inbox/research/
+```
+
+Project overview artefacts go in:
+
+```text
+hippocampus/inbox/project/
+```
+
+Generated review drafts are written to:
+
+```text
+hippocampus/imports/research/
+hippocampus/imports/project/
+```
+
+Promoted source data is written to:
+
+```text
+hippocampus/source/
+```
+
+## Triage First For Large Corpora
+
+A shared drive holds far more than one round's worth of material, and most of it is
+context rather than findings. Triage before you copy anything into the inbox:
+
+```bash
+npx hippocampus research triage -- --source "/path/to/research folder" --topic <topic>
+```
+
+This reads every file, ranks it by the signals that matter to the schema, maps it to
+the records it could become, and writes a review pack to
+`artefacts/research-triage/`. It copies nothing, imports nothing and writes nothing
+to `hippocampus/source/`.
+
+Work down the tiers, take Tier 1 into the inbox first, and follow the round workflow
+below one round at a time. `.github/skills/triage-research-corpus.skill.md` is the
+skill entry point.
+
+## Research Round Workflow
+
+1. Drop the artefact into `hippocampus/inbox/research/`. PowerPoint, PDF, Word,
+   Excel, CSV, plain text and `.url` shortcuts are all read.
+2. Run:
+
+   ```bash
+   npx hippocampus research sync -- --no-promote
+   ```
+
+3. Find the generated draft:
+
+   ```text
+   hippocampus/imports/research/<draft-id>.review.md
+   hippocampus/imports/research/<draft-id>.json
+   ```
+
+4. Ask an AI coding agent to use `.github/agents/ingestion-curator.md` and follow `.github/skills/structure-research-draft.skill.md` against the draft.
+5. Review the changed JSON and the review Markdown together.
+6. Run a dry-run promotion:
+
+   ```bash
+   npx hippocampus promote research -- --draft hippocampus/imports/research/<draft-id>.json --dry-run
+   ```
+
+7. If the draft is still weak, ask AI to revise the JSON again. Do not promote weak drafts.
+8. Promote only after review:
+
+   ```bash
+   npx hippocampus promote research -- --draft hippocampus/imports/research/<draft-id>.json
+   ```
+
+9. Validate the knowledge base:
+
+   ```bash
+   npx hippocampus validate
+   ```
+
+## Project Context Workflow
+
+Use project context for overview material such as project purpose, pilot scope, service proposition, policy context, constraints, channels, milestones and source facts.
+
+1. Drop the deck or text-based PDF into `hippocampus/inbox/project/`.
+2. Run:
+
+   ```bash
+   npx hippocampus import project-context -- --no-promote
+   ```
+
+3. Find the generated draft:
+
+   ```text
+   hippocampus/imports/project/<draft-id>.review.md
+   hippocampus/imports/project/<draft-id>.json
+   ```
+
+4. Ask an AI coding agent to use `.github/agents/ingestion-curator.md` and follow `.github/skills/structure-project-context-draft.skill.md` against the draft.
+5. Review the changed JSON and the review Markdown together.
+6. Run a dry-run promotion:
+
+   ```bash
+   npx hippocampus promote project-context -- --draft hippocampus/imports/project/<draft-id>.json --dry-run
+   ```
+
+7. Promote only after review:
+
+   ```bash
+   npx hippocampus promote project-context -- --draft hippocampus/imports/project/<draft-id>.json
+   ```
+
+8. Validate the knowledge base:
+
+   ```bash
+   npx hippocampus validate
+   ```
+
+## AI Prompt Rules
+
+Use a strong reasoning model for structuring drafts.
+
+The AI must:
+
+- use only the extracted review Markdown and existing JSON draft
+- preserve extracted text, source paths and import metadata
+- return actual JSON changes, not a summary of intended changes
+- leave weak or missing material out rather than inventing it
+- keep warnings for claims that need human review
+- cite source page, slide, heading or section where the schema has a suitable field
+- avoid generic policies, dates, risks or users that are not named in the source
+
+Do not accept a report that says the model improved the file. Check the diff.
+
+## Review Checklist
+
+Before promotion, check:
+
+- The draft file actually changed.
+- The extracted text supports every important claim.
+- Research insights are genuine findings, not slide headings.
+- Participants are real people, roles or user groups from the source, not section labels.
+- Design actions are concrete and traceable to evidence.
+- Project context facts are not table-of-contents entries.
+- Risks are complete statements, not fragments containing the word "risk".
+- Duplicates and contradictions are called out.
+- Schema dry-run passed.
+- A human has approved the draft.
+- Insight counts are plausible. The heading-scoped pass is deliberately generous: it
+  offers candidates, and an artefact yielding well over 60 insights is capturing
+  narrative text that a human should cut back before promotion.
+
+## What Good Looks Like
+
+A good promoted research round should let a team answer:
+
+- What did we learn?
+- Who did we learn it from?
+- What evidence supports it?
+- What design or product action follows?
+- Which users, needs, journeys, screens or decisions does it affect?
+
+A good promoted project context draft should let a team answer:
+
+- What is this service or pilot trying to do?
+- Who is involved?
+- What constraints, policies, channels and risks matter?
+- What milestones or facts are supported by source material?
+- Which artefacts prove those claims?
+
+## Recommended Team Operating Model
+
+For each research round:
+
+1. Researcher drops the final deck/report into the inbox.
+2. Designer or researcher runs the sync command.
+3. AI structures the generated draft.
+4. Researcher checks evidence and wording.
+5. Designer checks graph links and product implications.
+6. One person promotes the draft.
+7. The team reviews `/hippocampus`, `/hippocampus/graph` and `/hippocampus/trace`.
+
+This keeps Hippocampus repeatable: teams can use AI heavily, but the graph only changes through reviewable drafts and validation scripts.