@jterrats/open-orchestra 1.0.11 → 1.0.12

package/docs/backlog/web-code-editor-lsp-spike.md

@@ -0,0 +1,289 @@
+# Spike: Orchestra-Aware Web Code Editor With LSP
+
+Backlog Item ID: EPIC-ORCHESTRA-AWARE-CODE-EDITOR
+GitHub Issue: GH-447
+Lead role: Architect
+Supporting roles: UX/UI, Developer, Security, QA
+Status: proposed
+
+## Goal
+
+Define the MVP architecture for an embedded web code editor that lets users
+inspect workspace-local files, use standard language intelligence, and perform
+governed edits without losing Orchestra task traceability, policy gates, QA
+evidence, or workspace isolation.
+
+This spike is architecture and planning only. It does not implement source
+changes.
+
+## Product Position
+
+The editor should not compete with VS Code as a full IDE. Its differentiator is
+workflow-aware editing: active task, backlog item, acceptance criteria, role
+contracts, context-pack suggestions, visible policy gates, task-scoped diffs,
+file timeline, rollback guidance, and quality warnings such as god-file risk,
+missing tests, stale generated docs, and release gate blockers.
+
+## Editor Choice
+
+Use CodeMirror 6 for the MVP, with Monaco kept as a later option if native VS
+Code-language parity becomes more valuable than bundle control and UI
+composition.
+
+CodeMirror 6 benefits: smaller, modular, easier to compose inside the existing
+React/Vite console, more controllable for mobile and constrained panels, and
+well suited to incremental viewer, diagnostics, diff preview, and governed-save
+slices. Costs: LSP support is less turnkey than Monaco and users expecting VS
+Code behavior may notice missing commands.
+
+Monaco benefits: closer to VS Code expectations with mature TypeScript and
+JavaScript language behavior, diagnostics, hover, completion, symbols, and
+go-to-definition. Costs: heavier runtime, worker/CSP complexity, harder mobile
+composition, and a stronger pull toward an IDE-like footprint.
+
+Adopt CodeMirror 6 for the first implementation slices. Revisit Monaco after
+the read-only viewer, diff integration, and governed write contract are proven.
+The LSP bridge contract should be editor-neutral so Monaco can be introduced
+later without changing policy, workspace, or evidence boundaries.
+
+## Architecture Boundary
+
+```mermaid
+flowchart LR
+  ui["Web console editor shell"]
+  context["Orchestra context sidebar"]
+  api["Editor API contract"]
+  policy["Policy gate service"]
+  workspace["Workspace file service"]
+  git["Git diff and timeline service"]
+  lsp["LSP bridge supervisor"]
+  server["Language server process"]
+  evidence["Evidence and review records"]
+
+  ui --> api
+  context --> api
+  api --> policy
+  api --> workspace
+  api --> git
+  api --> lsp
+  lsp --> server
+  policy --> evidence
+  workspace --> evidence
+  git --> evidence
+```
+
+The web console must remain a client of stable JSON contracts. The browser does
+not read arbitrary files directly, spawn processes, or decide write policy.
+
+## LSP Bridge Contract
+
+The LSP bridge is a local or SaaS-side service boundary that mediates language
+server lifecycle, document sync, diagnostics, and language requests.
+
+Required responsibilities:
+
+- accept only workspace-relative paths that resolve inside the active workspace;
+- require task id and editor session id for every opened document;
+- start language servers through array-based process APIs, never shell
+  interpolation;
+- keep one supervised language-server pool per workspace, language, and tenant
+  boundary;
+- enforce startup timeout, idle shutdown, max memory, max files, max document
+  size, and bounded request concurrency;
+- redact or suppress diagnostics that expose internal absolute paths or secret
+  values;
+- degrade to syntax-only editing when a language server is unavailable;
+- emit lifecycle and diagnostic summaries that can be attached as evidence.
+
+Initial target: TypeScript and JavaScript through the project-local TypeScript
+server or a pinned LSP-compatible wrapper. Extension targets are Python
+through pyright, Java through jdtls, .NET through OmniSharp or Roslyn LSP, Apex
+through a Salesforce language server if licensing and setup allow it, and custom
+domain servers through future extension manifests.
+
+LSP messages must never be trusted as policy input. They can inform diagnostics
+and navigation, but write permission remains owned by the policy gate service.
+
+## Workspace Isolation
+
+All file operations must be scoped to an active workspace root resolved by the
+server.
+
+Local mode:
+
+- bind local services to `127.0.0.1`;
+- reject absolute paths, traversal, symlinks that escape the workspace, and
+  generated or secret-sensitive paths unless explicitly allowlisted;
+- use existing task and evidence state under `.agent-workflow/` as the durable
+  traceability source;
+- keep language servers local to the workspace and terminate them when the
+  workspace session closes.
+
+SaaS mode:
+
+- isolate tenants by account, workspace, runtime sandbox, storage prefix, and
+  language-server process boundary;
+- enforce data residency before source text or diagnostics cross regions;
+- avoid shared language-server processes across tenants;
+- store only policy-approved evidence summaries unless the user explicitly
+  attaches file snippets or diffs;
+- require audit records for open, preview, save request, policy decision, write,
+  rollback, and evidence attachment events.
+
+## Governed Edit Contract
+
+MVP starts read-only. Edit mode is a deliberate transition with visible task and
+policy state.
+
+Open file request fields: `taskId`, `workspaceId`, `relativePath`, `mode`
+(`readOnly` or `editIntent`), and optional `selection`, `contextPackId`, and
+`role`.
+
+Save request fields: `taskId`, `workspaceId`, `relativePath`, `baseRevision`,
+`proposedContentHash`, `patch`, `userIntent`, and `policyAcknowledgements`.
+
+Save response fields: `decision` (`allowed`, `blocked`, or `needsReview`),
+`diffSummary`, `policyFindings`, `evidenceId`, `rollbackHint`, and
+`nextActions`.
+
+Policy checks before write:
+
+- task exists, has backlog item, and is active or explicitly selected;
+- path is inside owned write scope for the task;
+- file is not locked by another role or workflow;
+- file is not a release bump file unless the task permits release ownership;
+- generated files require source-of-truth confirmation;
+- security-sensitive paths require Security review;
+- save is based on current file revision or requires conflict resolution;
+- diff preview is acknowledged before write.
+
+Every write must create or update a task-scoped diff record and evidence hook.
+The UI should offer rollback guidance based on Git state, but it should not
+perform destructive Git operations without a separate approval flow.
+
+## UX Flow
+
+Primary user: a human operator reviewing or making a governed task-scoped edit
+from the local or SaaS web console.
+
+MVP flow: user selects an active task; the console shows acceptance criteria,
+owned paths, gates, related files, and evidence gaps; the user opens a related
+file in read-only mode; the editor shows syntax highlighting, diagnostics when
+available, and a task context sidebar; the user requests edit mode; the console
+shows write policy status, locks, and required reviewers; the user edits,
+previews a diff, submits a save request, receives an allowed, blocked, or
+needs-review decision, and gets an evidence link plus next steps.
+
+Responsive behavior:
+
+- mobile defaults to task context, file list, and read-only code view with
+  collapsible diagnostics;
+- tablet uses stacked editor and context panels;
+- desktop uses three regions: file/navigation rail, editor/diff, and Orchestra
+  context sidebar;
+- no critical action should depend on horizontal scrolling;
+- keyboard users can open files, search, inspect diagnostics, preview diff,
+  request edit mode, and submit or cancel saves.
+
+Required states:
+
+- loading file and loading diagnostics;
+- empty related files;
+- language server unavailable with syntax-only fallback;
+- policy blocked with clear next action;
+- conflict detected with reload or compare options;
+- save succeeded with evidence link;
+- save failed without exposing stack traces or internal paths.
+
+## Security Constraints
+
+Security review is mandatory before implementation because this feature touches
+file paths, process execution, workspace source code, secrets, network calls,
+and future multi-tenant boundaries.
+
+Non-negotiables:
+
+- no shell interpolation for language server startup or Git operations;
+- validate and normalize every path server-side;
+- never expose stack traces, host paths, environment variables, or raw process
+  errors to the browser;
+- scan proposed diffs for configured secret patterns before write;
+- treat workspace files and LSP responses as untrusted input;
+- sanitize markdown, diagnostics, and hover content before rendering;
+- apply content security policy before enabling Monaco workers or remote
+  extension assets;
+- fail closed when policy, path validation, revision checks, secret scan, or
+  audit write fails;
+- keep local ports bound to `127.0.0.1` by default;
+- require explicit tenant and data residency controls before SaaS rollout.
+
+## QA Evidence Strategy
+
+The spike output is validated by review evidence. Implementation stories need
+observable acceptance evidence.
+
+Recommended automated coverage:
+
+- unit tests for path normalization, task write-scope policy, save contract,
+  revision conflict handling, and LSP lifecycle state transitions;
+- contract tests for editor API request and response shapes;
+- integration tests with fake language-server processes for diagnostics,
+  timeout, crash, unavailable server, and idle shutdown;
+- Playwright tests for read-only viewer, responsive layout, keyboard flow,
+  policy-blocked save, conflict recovery, and successful evidence link;
+- security tests for traversal, symlink escape, secret diff rejection, unsafe
+  diagnostic rendering, and blocked process arguments.
+
+Evidence required per implementation story:
+
+- exact commands and pass/fail results;
+- AC-to-evidence matrix;
+- screenshots or traces for desktop and mobile editor flows;
+- sample diff/evidence artifact;
+- security review result for file/process/network changes;
+- documented deferred language support when a language server is unavailable.
+
+## Implementation Slices
+
+1. Read-only code viewer and file open contract: CodeMirror 6 surface, syntax
+   highlighting, loading/empty/error states, path validation, task association,
+   and Playwright smoke.
+2. Task context sidebar: active task, backlog item, acceptance criteria, roles,
+   owned paths, context-pack references, evidence gaps, and gate warnings.
+3. Git diff and task timeline integration: revision metadata, diff preview,
+   task-scoped file timeline, conflict detection, and evidence hook.
+4. TypeScript/JavaScript LSP bridge MVP: supervised lifecycle, diagnostics,
+   hover, completion, go-to-definition, fallback states, and fake LSP tests.
+5. Governed edit mode: edit intent, patch submission, policy checks, diff
+   acknowledgement, save response, blocked UX, and needs-review UX.
+6. Security hardening: traversal and symlink protection, secret scanning,
+   process allowlist, diagnostic sanitization, CSP review, and local binding
+   checks.
+7. SaaS isolation design: tenant sandboxing, process isolation, storage and data
+   residency policy, audit logs, quota controls, and abuse controls.
+8. Language extension framework: server capability manifest, health status,
+   per-language setup guidance, and extension points.
+
+## Open Risks
+
+- LSP servers execute project-aware code paths and can be expensive or unsafe if
+  not tightly supervised.
+- Monaco may be required later if CodeMirror extension quality does not meet
+  user expectations for TypeScript-heavy projects.
+- SaaS editing has materially higher tenant isolation, data residency, and audit
+  requirements than local mode.
+- Secret detection can produce false negatives and should not be the only
+  control before writing.
+- Generated-file editing can violate source-of-truth contracts unless policy
+  gates are strict.
+
+## Recommended Next Stories
+
+- GH-447-A: Build read-only CodeMirror file viewer with task-scoped open
+  contract.
+- GH-447-B: Add Orchestra task context sidebar for editor sessions.
+- GH-447-C: Add Git diff preview and task-scoped file timeline.
+- GH-447-D: Implement TypeScript/JavaScript LSP bridge MVP with fake LSP tests.
+- GH-447-E: Implement governed edit mode with policy-gated saves.
+- GH-447-F: Add editor security hardening and negative test matrix.
+- GH-447-G: Define SaaS tenant isolation and data residency ADR.

package/docs/context-vault.md

@@ -0,0 +1,80 @@
+# Context Vault
+
+Context vault is the planned workspace catalog for source materials that inform
+agent work but should not be pasted directly into every prompt. It covers
+documents, statements of work, PDFs, diagrams, images, audio/video recordings,
+transcripts, and client reference artifacts.
+
+The vault is related to context indexing and transcription, but it has a
+different job: it records provenance, sensitivity, retention, ownership, and
+safe consumption rules for project inputs.
+
+## Goals
+
+- Register source artifacts with stable metadata and checksums.
+- Classify sensitivity before an artifact is used by an agent.
+- Convert large or binary inputs into bounded summaries, excerpts, transcripts,
+  or context packs.
+- Preserve provenance so BA, PO, Architect, QA, and Release can cite which
+  artifact informed a requirement, risk, decision, or validation result.
+- Keep raw sensitive inputs out of runtime prompts unless policy explicitly
+  allows a bounded excerpt.
+
+## Artifact Metadata
+
+Every vault artifact should record:
+
+- artifact id and workspace or tenant id;
+- original source, file name, media type, size, checksum, and registered time;
+- owner role or user;
+- sensitivity classification and retention policy;
+- ingestion status, conversion status, redaction status, and error state;
+- derived outputs such as Markdown conversion, transcript, summary, embeddings,
+  or context-pack references.
+
+## Consumption Model
+
+Agents should not load raw vault artifacts by default. They should request a
+bounded context pack or artifact summary scoped to the active task, role, phase,
+and token budget.
+
+The pack should include:
+
+- artifact references and provenance links;
+- selected excerpts with inclusion reasons;
+- redaction decisions and omitted-sensitive-data notes;
+- budget summary and truncation indicators;
+- stale or failed-ingestion warnings.
+
+## Security And Privacy
+
+Vault ingestion must be tenant-aware and fail closed when classification or
+redaction is uncertain. Secrets, credentials, health data, payment data, and
+other regulated content require explicit policy handling before they can be
+summarized or exposed to a runtime.
+
+External conversion or transcription providers are opt-in. Local conversion and
+local transcription should be preferred when a workspace or tenant requires
+offline handling.
+
+## API Shape
+
+The implementation should define APIs before storage details:
+
+- `vault artifact add` or API equivalent for registering files and metadata;
+- `vault artifact list` with pagination, filtering, status, type, and
+  sensitivity filters;
+- `vault artifact show` for metadata and derived-output status;
+- `vault ingest` for conversion, transcription, and redaction jobs;
+- `context pack build` integration that can cite vault artifacts without
+  reading raw files broadly.
+
+The web console can later expose this as a catalog view, but it should consume
+the same domain services as the CLI/API.
+
+## Related Work
+
+- GitHub issue `#449`: context vault epic.
+- GitHub issue `#367`: audio and video transcription evidence skill.
+- GitHub issues `#423` through `#427`: context index, search, bounded context
+  packs, redaction, and runtime prompt integration.

package/docs/site-manifest.json

@@ -117,6 +117,7 @@
       { "title": "Sonar quality gates", "source": "docs/sonar-quality-gates.md", "heading": "Sonar Quality Gates" },
       { "title": "Sonar architecture model", "source": "docs/sonar-architecture-model.md", "heading": "Sonar Architecture Model" },
       { "title": "Runtime adapters", "source": "docs/runtime-adapters.md", "heading": "Runtime Adapters" },
+      { "title": "Context vault", "source": "docs/context-vault.md", "heading": "Context Vault" },
       { "title": "Site content workflow", "source": "docs/site-content-workflow.md", "heading": "Public Site Content Workflow" }
     ]
   },

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@jterrats/open-orchestra",
-  "version": "1.0.11",
+  "version": "1.0.12",
   "type": "module",
   "workspaces": [
     "extensions/vscode-open-orchestra",