@a5c-ai/krate 5.0.1-staging.f672fe79b

package/docs/agents/memory-context-integration-spec.md

@@ -0,0 +1,194 @@
+# Memory context integration spec
+
+## Purpose
+
+This document defines how Krate's context layer should read the company brain and add memory-backed context to `AgentContextBundle` resources. Memory follows the same core rules as every other context source: permission review, provenance, redaction, bounded size, immutable digests, and preview before dispatch.
+
+## Source families
+
+| Family | Example | Retrieval | Prompt treatment |
+| --- | --- | --- | --- |
+| Graph YAML records | `graph/orgs/[org]/repositories/krate.yaml` | ID lookup, kind filters, edge traversal | summarized records with IDs, owners, and edge breadcrumbs. |
+| Markdown records | `runbooks/ci/playwright-flake.md` | frontmatter filters and bounded reads | title, frontmatter, body excerpts, source path. |
+| Free-form Markdown | `notes/investigations/*.md` | grep/ripgrep search within allowed paths | line excerpts with context and digest. |
+| Ontology records | `ontology/node-kinds.yaml` | schema lookup and validation report | validation context, not task facts unless requested. |
+
+Graph and grep results should work together: graph traversal can identify relevant repositories, services, decisions, and runbooks; grep can then search those candidate paths for specific failure signatures or terms.
+
+## `AgentContextBundle` additions
+
+```yaml
+spec:
+  memory:
+    enabled: true
+    repositoryRef: org-company-brain
+    requestedRef: main
+    resolvedCommit: abcdef1234567890
+    refResolution:
+      mode: current
+    snapshotRef: memory-snapshot-01hx
+    queryManifestDigest: sha256:...
+    ontologyDigest: sha256:...
+    indexDigest: sha256:...
+    sources:
+      - kind: memory-graph-record
+        id: runbook:ci-playwright-flake
+        path: runbooks/ci/playwright-flake.md
+        digest: sha256:...
+      - kind: memory-grep-excerpt
+        path: notes/investigations/2026-05-playwright.md
+        lineStart: 42
+        lineEnd: 48
+        digest: sha256:...
+    limits:
+      maxMemoryBytes: 128000
+      truncated: false
+```
+
+Memory entries may also appear in the existing `sources` list, but the dedicated `memory` block is required for retries, stale-memory warnings, historical refs, and audits.
+
+## Query modes
+
+| Mode | Behavior | Use |
+| --- | --- | --- |
+| `graph-only` | query structured graph/frontmatter records only | ownership, dependency, runbook, and ontology-driven tasks. |
+| `grep-only` | search allowed Markdown paths without graph traversal | raw notes, broad recall, and unstructured history. |
+| `graph-and-grep` | graph narrows candidate paths, then grep searches them | default for repository tasks. |
+| `document-read` | read selected memory documents by path or ID | user-selected context in composer. |
+| `ontology-validation` | validate proposed graph/frontmatter changes | update review and memory CI. |
+| `diff-ref` | compare current memory with pinned memory | time-travel and stale-context warnings. |
+
+## Ref resolution
+
+```yaml
+memory:
+  refResolution:
+    mode: current | explicit-ref | ref-at-time | snapshot-tag
+    requested: main
+    resolvedCommit: abcdef1234567890
+    resolvedAt: 2026-05-10T12:00:00Z
+```
+
+Rules:
+
+- `current` resolves the configured default branch at dispatch creation.
+- `explicit-ref` resolves a branch, tag, or SHA.
+- `ref-at-time` resolves the latest approved commit at or before a timestamp.
+- `snapshot-tag` resolves a Krate-created stable snapshot tag.
+- Ambiguous or missing refs block dispatch before Agent Mux launch.
+
+## Assembly flow
+
+```text
+source event or manual dispatch
+  -> resolve AgentStack and AgentMemorySource
+  -> review memory read/query permissions
+  -> resolve memory ref to commit
+  -> load ontology and index manifests at commit
+  -> run graph, frontmatter, and grep retrieval
+  -> redact, rank, and bound selected memory
+  -> create AgentMemorySnapshot and AgentMemoryQuery records
+  -> add memory block to AgentContextBundle
+  -> launch AgentDispatchAttempt with immutable snapshot
+```
+
+## Ranking requirements
+
+The preview must explain why each memory item was selected. Rank by:
+
+1. explicit user selection;
+2. direct association to repository, issue, PR, service, stack, trigger, or skill;
+3. graph edge distance from source refs;
+4. allow-list specificity;
+5. grep match strength and recency;
+6. owner-approved status;
+7. size and freshness constraints.
+
+## Prompt rendering
+
+Memory should render in a dedicated prompt section after repository instructions and before transient logs or diffs:
+
+```text
+## Company Brain Memory
+Memory repository: org-company-brain
+Resolved commit: abcdef1234567890
+Query manifest: sha256:...
+
+### Graph records
+- runbook:ci-playwright-flake ...
+
+### Markdown excerpts
+- notes/investigations/2026-05-playwright.md:42 ...
+```
+
+The prompt must tell agents that memory is advisory and may be stale, especially when pinned to a historical ref.
+
+## Historical memory runs
+
+When dispatch uses `refAt` or an old snapshot:
+
+- UI displays `Memory is pinned to <commit> from <date>`.
+- Context bundle stores current memory commit separately for comparison.
+- Run detail offers `Diff memory against current`.
+- Agent prompt includes a stale-memory banner.
+- Memory read tools default to the pinned commit.
+- Memory update proposals target current `main` unless policy explicitly says otherwise.
+
+Example:
+
+```yaml
+memory:
+  requestedRefAt: 2026-05-08T09:30:00Z
+  resolvedCommit: 13579bdf2468
+  currentCommitAtDispatch: abcdef1234567890
+  staleBy: 2d3h
+```
+
+## Memory tools
+
+After Krate permission review, Agent Mux can expose:
+
+- `memory.graph.search`: search graph records by text, kind, edge, owner, or association.
+- `memory.record.read`: read graph or Markdown records by ID/path at the pinned commit.
+- `memory.docs.grep`: grep allowed Markdown paths at the pinned commit.
+- `memory.snapshot.diff`: diff pinned memory against another ref.
+- `memory.update.propose`: create a proposed memory patch artifact.
+- `memory.ontology.validate`: validate proposed graph/frontmatter changes.
+
+Tools operate against the dispatch memory snapshot by default. Reading current memory from a historical run requires explicit refresh or approval.
+
+## Failure behavior
+
+| Failure | Required behavior |
+| --- | --- |
+| Memory repo unavailable | block if memory is required; warn and continue only if optional. |
+| Ref cannot resolve | block dispatch. |
+| Ontology invalid | block update merges; allow reads only with warning if policy permits. |
+| Grep returns too much | truncate with omitted-count summary. |
+| Permission denied | omit content and show denied path/kind without leaking values. |
+| Secret-like content detected | redact; mark unsafe if redaction is too broad. |
+
+## Acceptance criteria
+
+- `AgentContextBundle` explains memory repository, requested ref, resolved commit, selected records, selected excerpts, and query manifest.
+- Graph and grep results can be combined in one dispatch.
+- Time-travel memory uses a commit-pinned snapshot and never silently refreshes during retry.
+- Agents can only call memory tools for paths/kinds granted by Krate.
+- Context preview and run detail expose stale-memory warnings and diff actions.
+
+## Babysitter memory context
+
+Context assembly can include curated Babysitter run memory from the org company brain:
+
+- `babysitter/MEMORY.md` for stable orchestration instructions and conventions;
+- session summaries for previous related agent work;
+- curated run journals for replaying decisions and state transitions;
+- task results and artifact manifests for evidence-backed context;
+- retrospectives for process improvements and known pitfalls.
+
+Selection rules:
+
+- match org first, then repository, stack, process, trigger, issue/PR, and run status;
+- prefer summarized sessions and retrospectives over raw journal events unless a replay/debug task needs detail;
+- include raw journal excerpts only with line/event bounds and redaction;
+- pin all imported run memory to the same memory commit as other company brain sources.

package/docs/agents/memory-ontology-schema-spec.md

@@ -0,0 +1,253 @@
+# Memory ontology and file schema spec
+
+## Purpose
+
+The company brain needs a maintained ontology so shared memory stays navigable, enforceable, searchable, and useful to agents. This document defines Markdown/YAML schema conventions for graph records, frontmatter records, free-form documents, ontology files, and generated indexes.
+
+## Principles
+
+- Human-readable files are authoritative.
+- Canonical facts have stable IDs.
+- Graph structure improves retrieval but does not block useful notes.
+- Ontology changes are reviewed like code.
+- Derived indexes are reproducible from Git contents.
+- Memory records expose owners, status, and source references where possible.
+
+## File classes
+
+| Class | Pattern | Structure |
+| --- | --- | --- |
+| Graph records | `graph/**/*.yaml` | Atlas-style `nodeKind`, `id`, `attributes`, `edges`. |
+| Markdown records | `runbooks/**/*.md`, `decisions/**/*.md`, `incidents/**/*.md`, `repositories/**/*.md` | YAML frontmatter plus body. |
+| Free-form notes | `notes/**/*.md`, `meetings/**/*.md`, `scratch/**/*.md` | body required; frontmatter optional. |
+| Ontology | `ontology/**/*.yaml` | node kinds, edge kinds, vocabularies, validation rules. |
+| Generated indexes | `indexes/**/*` | generated reports tied to source commit/digest. |
+
+## Graph YAML schema
+
+```yaml
+nodeKind: Runbook
+id: runbook:ci-playwright-flake
+attributes:
+  title: Playwright flake triage
+  status: approved
+  owners: [team:platform]
+  summary: How to diagnose recurring Playwright failures in Krate CI.
+  tags: [ci, playwright, krate]
+  updatedAt: 2026-05-10T12:00:00Z
+edges:
+  applies_to_repo:
+    - target: repository:krate
+  owned_by:
+    - target: team:platform
+  supersedes:
+    - target: runbook:old-playwright-flake
+```
+
+Required graph fields:
+
+- `nodeKind`: ontology kind name.
+- `id`: stable prefixed ID.
+- `attributes.title`: display name.
+- `attributes.status`: `draft`, `approved`, `deprecated`, or `archived`.
+- `attributes.owners`: one or more teams/users.
+- `attributes.updatedAt`: ISO timestamp.
+
+## Markdown frontmatter schema
+
+```markdown
+---
+id: decision:agent-memory-git-backed
+kind: Decision
+title: Use Git as source of truth for company brain
+status: approved
+owners:
+  - team:platform
+aliases:
+  - company brain git memory
+repoRefs:
+  - repository:krate
+tags:
+  - agents
+  - memory
+related:
+  documents:
+    - runbook:agent-memory-update-review
+  supersedes:
+    - decision:agent-memory-db-only
+updatedAt: 2026-05-10T12:00:00Z
+---
+
+# Use Git as source of truth for company brain
+
+Decision body...
+```
+
+| Field | Required | Notes |
+| --- | --- | --- |
+| `id` | yes for records | Stable graph ID; optional for pure free-form notes. |
+| `kind` | yes for records | Ontology node kind. |
+| `title` | yes | Display title; free-form notes can fall back to first H1. |
+| `status` | yes for records | `draft`, `approved`, `deprecated`, `archived`. |
+| `owners` | yes for records | Team/user IDs. |
+| `aliases` | no | Search and disambiguation terms. |
+| `repoRefs` | no | Associated Krate repositories. |
+| `tags` | no | Controlled vocabulary preferred. |
+| `related` | no | Edge map from edge kind to target IDs. |
+| `updatedAt` | yes for records | ISO timestamp. |
+| `sensitivity` | no | `public`, `internal`, `restricted`, `secret-metadata`. |
+| `sourceRefs` | no | Issues, PRs, dispatch runs, incidents, docs, URLs. |
+
+## Free-form Markdown
+
+Free-form files are intentionally lightweight:
+
+```markdown
+---
+title: Investigation notes for flaky Krate checks
+owners: [user:alice]
+tags: [investigation, ci]
+repoRefs: [repository:krate]
+status: draft
+---
+
+Raw notes...
+```
+
+Rules:
+
+- frontmatter is optional but recommended;
+- `id` is optional unless the note should be graph-addressable;
+- grep search may return excerpts from files without IDs;
+- promotion from note to canonical record should preserve source links.
+
+## Initial node kinds
+
+| Node kind | Purpose |
+| --- | --- |
+| `Organization` | company or business unit. |
+| `Team` | owner, reviewer, or operating group. |
+| `Repository` | Krate repository identity and associated practices. |
+| `Service` | deployed service or component. |
+| `Package` | package/library/module ownership and practices. |
+| `Runbook` | operational procedure. |
+| `Decision` | architectural, product, or process decision. |
+| `Incident` | incident summary, remediation, follow-up. |
+| `AgentPractice` | reusable lesson for agent dispatches. |
+| `Skill` | agent skill knowledge and requirements. |
+| `Tool` | native tool or MCP capability knowledge. |
+| `Customer` | customer-specific context when allowed. |
+| `ProductArea` | product/domain grouping. |
+| `Term` | glossary, aliases, and terminology. |
+| `PromptFragment` | reviewed context text reusable by stacks. |
+
+## Initial edge kinds
+
+| Edge kind | Use |
+| --- | --- |
+| `documents` | page or record documents another node. |
+| `implements` | service/package implements a decision or capability. |
+| `depends_on` | dependency or operational prerequisite. |
+| `supersedes` | replaces older record. |
+| `owned_by` | ownership by team/user. |
+| `applies_to_repo` | memory applies to a repository. |
+| `applies_to_stack` | memory applies to an agent stack. |
+| `mentions` | weak mention/reference. |
+| `derived_from` | extracted from run, issue, PR, incident, or note. |
+| `requires_secret` | tool/skill/runbook requires a named secret grant. |
+| `requires_config` | tool/skill/runbook requires a named config grant. |
+| `safe_for_trigger` | approved for a trigger source or trust level. |
+| `resolved_by` | incident or issue resolved by runbook, decision, PR, or dispatch. |
+
+## ID conventions
+
+| Kind | Example |
+| --- | --- |
+| Repository | `repository:krate` |
+| Team | `team:platform` |
+| Runbook | `runbook:ci-playwright-flake` |
+| Decision | `decision:agent-memory-git-backed` |
+| Incident | `incident:2026-05-krate-ci-outage` |
+| AgentPractice | `agent-practice:prefer-focused-tests-first` |
+| Skill | `skill:focused-test-selection` |
+| Tool | `tool:memory-docs-grep` |
+
+IDs are immutable. Renames update title and aliases. Replacements use `supersedes` and deprecate the older record.
+
+## Validation rules
+
+Validators should check:
+
+- YAML parse errors and Markdown frontmatter parse errors;
+- duplicate IDs;
+- unknown node kinds and edge kinds;
+- missing required fields;
+- invalid owner IDs;
+- dangling edges;
+- forbidden status transitions;
+- forbidden secrets or high-entropy strings;
+- path policy violations;
+- stale generated indexes;
+- ontology compatibility version.
+
+## Derived index shape
+
+```yaml
+generatedAt: 2026-05-10T12:00:00Z
+sourceCommit: abcdef1234567890
+ontologyDigest: sha256:...
+stats:
+  records: 1200
+  edges: 4200
+  markdownRecords: 550
+  freeFormDocuments: 900
+  parseErrors: 0
+records: {}
+edges: []
+pathIndex: {}
+ownerIndex: {}
+repoIndex: {}
+tagIndex: {}
+```
+
+Indexes may be committed for review or stored as controller artifacts, but Krate must be able to rebuild them from source.
+
+## Governance
+
+- Ontology changes require memory-owner review.
+- New node/edge kinds need examples and validation rules.
+- Deprecated kinds stay readable until migration completes.
+- Reports show unowned records, stale records, dangling edges, and sensitive records.
+- Canonical records should include source references when derived from runs, incidents, issues, or PRs.
+
+## Acceptance criteria
+
+- A developer can add a useful free-form note without learning the full graph schema.
+- A memory steward can promote a note into a canonical graph or Markdown record.
+- Validators catch duplicate IDs, dangling edges, unknown kinds, and secret-like content.
+- Krate can build graph traversal, frontmatter filters, and grep search from the same Git ref.
+- UI can explain owners, source refs, and associations to repositories, stacks, skills, tools, triggers, and runs.
+
+## Babysitter memory schema
+
+Add ontology support for Babysitter orchestration memory:
+
+| Node kind | Purpose |
+| --- | --- |
+| `BabysitterRun` | org-scoped orchestration run with status, source repo, process, and task graph. |
+| `BabysitterSession` | chat/session summary linked to one or more dispatches or runs. |
+| `RunJournalEvent` | ordered event extracted from `.a5c/runs/<run>/journal`. |
+| `RunTaskResult` | task-level result, evidence, artifacts, and validation status. |
+| `RunRetrospective` | durable lesson or process improvement derived from a run. |
+
+Additional edge kinds:
+
+| Edge kind | Use |
+| --- | --- |
+| `has_journal_event` | run contains ordered journal event. |
+| `has_task_result` | run contains task result. |
+| `summarized_by` | session/run is summarized by Markdown memory. |
+| `produced_artifact` | task or run produced artifact manifest/digest. |
+| `learned_from` | practice, runbook, or retrospective derived from run/session. |
+
+`MEMORY.md` may remain a special entrypoint file, but durable facts extracted from it should use normal graph IDs and frontmatter when promoted.

package/docs/agents/memory-operations-runbook.md

@@ -0,0 +1,121 @@
+# Memory operations runbook
+
+## Purpose
+
+This runbook defines operational flows for bootstrapping, validating, querying, updating, rolling back, and time-traveling the company brain memory repository.
+
+## Bootstrap org memory
+
+1. Create `AgentMemoryRepository` with `managedByKrate=true`.
+2. Krate creates or adopts the Git repository.
+3. Seed `ontology/`, `graph/`, `pages/`, `notes/`, `runbooks/`, `decisions/`, `incidents/`, and `indexes/`.
+4. Seed base ontology with node kinds, edge kinds, statuses, sensitivity levels, and owner vocabulary.
+5. Create default `AgentMemorySource` policies per repository/team.
+6. Build initial indexes and validation report.
+7. Expose `/agents/memory` only to users with memory read permission.
+
+## Validate memory repository
+
+Validation should run on every memory PR, scheduled reconcile, and manual UI request.
+
+Required checks:
+
+- parse YAML and Markdown frontmatter;
+- enforce ontology schema;
+- verify graph IDs and edge targets;
+- verify owner/team references;
+- scan for secret-like content;
+- rebuild derived indexes;
+- compare generated indexes with committed indexes when committed indexes are enabled;
+- produce `ontology-report.json` and update `AgentMemoryOntology` status.
+
+## Dispatch with current memory
+
+1. User or trigger selects stack.
+2. Krate resolves default memory branch to commit.
+3. Context assembler runs allowed memory queries.
+4. Run detail stores `AgentMemorySnapshot` and selected context.
+5. Agent Mux launch receives prompt content plus memory tool descriptors.
+
+## Dispatch with memory from two days ago
+
+1. User selects `Memory ref: two days ago` in advanced dispatch settings.
+2. Krate converts the request to an absolute timestamp.
+3. Memory controller finds the latest approved commit at or before that timestamp.
+4. UI shows resolved commit and current-vs-pinned diff summary.
+5. `AgentContextBundle` stores both resolved historical commit and current commit.
+6. Agent prompt includes a stale-memory banner.
+7. Agent tools default to the pinned memory commit.
+
+Example:
+
+```yaml
+memory:
+  repositoryRef: org-company-brain
+  refAt: 2026-05-08T12:00:00Z
+  resolutionPolicy: latest-commit-before-or-at
+  requireApprovedCommit: true
+```
+
+## Propose memory update from a run
+
+1. Agent writes a memory update artifact with file changes and rationale.
+2. Krate validates the patch against ontology, path, owner, and redaction policy.
+3. Krate creates `AgentMemoryUpdate`.
+4. If allowed, Krate opens a PR or internal review branch.
+5. Reviewers inspect diff, source run, selected evidence, and validation report.
+6. Merge updates default branch and rebuilds indexes.
+7. Original run links to merged memory commit.
+
+## Recover from bad memory
+
+1. Identify bad commit, PR, or update record.
+2. Disable affected `AgentMemorySource` paths if needed.
+3. Revert or fix-forward in the memory repository.
+4. Rebuild indexes.
+5. Mark affected `AgentMemorySnapshot` records as `KnownBad` without mutating their content.
+6. Notify owners of dispatches that consumed the bad memory.
+7. Add a `Decision` or `Incident` record describing remediation when appropriate.
+
+## Rotate or move memory repository
+
+1. Create a new `AgentMemoryRepository` in disabled/read-only mode.
+2. Mirror Git contents and verify digest parity.
+3. Rebuild indexes from source.
+4. Update `AgentMemorySource` policies to point to the new repository.
+5. Run dry-run context assembly for representative stacks.
+6. Switch writes after validation.
+7. Keep old repository read-only until retention expires.
+
+## Operational dashboards
+
+`/agents/memory` should show:
+
+- current commit and last successful index build;
+- ontology validation state;
+- pending updates and stale PRs;
+- top memory consumers by repository/stack;
+- recent historical-memory runs;
+- denied memory queries;
+- records without owners;
+- stale approved records;
+- secret-scan alerts.
+
+## Alerts
+
+| Alert | Severity | Response |
+| --- | --- | --- |
+| memory index build failed | warning/critical by duration | inspect parse errors and block new writes if stale. |
+| ontology validation failed on main | critical | disable update merges and surface degraded context warning. |
+| secret-like content detected | critical | block merge, revoke if leaked, notify owners. |
+| memory repo unreachable | warning | block required-memory dispatches; allow optional-memory dispatches with warning. |
+| stale generated indexes | warning | rebuild and compare source commit. |
+| historical ref cannot resolve | warning | block requested dispatch. |
+
+## Acceptance criteria
+
+- Operators can bootstrap an org memory repo from UI or CRD.
+- Every memory PR receives validation output before merge.
+- Users can run with current, explicit-ref, snapshot-tag, or ref-at-time memory.
+- Bad memory can be reverted without corrupting past run snapshots.
+- Dashboards make memory health, permissions, and pending updates visible.