@jterrats/open-orchestra 1.0.11 → 1.0.12

package/docs/audio-video-transcription-skill.md

@@ -1,58 +1,454 @@
-# Audio/Video Transcription Skill
+# Audio/Video Transcription Skill Spike
 
-Open Orchestra can treat recordings as workflow evidence through the
-`audio-video-transcription` skill. The skill is loaded only when task text,
-risks, evidence, or runtime prompts mention transcription, audio, video,
-recordings, interviews, sprint reviews, support calls, subtitles, VTT, or SRT.
+Task: `GH-367-TRANSCRIPTION-SKILL-SPIKE`
+Issue: `GH-367` - Audio and video transcription evidence skill
+Status: proposed architecture spike
+Lead role: Architect
+Review roles covered: Security/Privacy, QA, Developer
 
-## Privacy Defaults
+## Goal
 
-- Media is sensitive by default.
-- Local/offline transcription is the default path when an approved local engine
-  is available.
-- External transcription providers require explicit policy opt-in before media
-  or raw transcript text leaves the workspace.
-- Transcript artifacts must redact secrets, tokens, credentials, configured PII,
-  and regulated data markers before persistence.
-- Regulated contexts should record consent and retention notes before release.
+Define a local-first transcription skill that turns workflow-local audio and
+video artifacts into searchable, reviewable evidence without uploading media or
+raw transcript text by default.
 
-## Evidence Contract
+The skill should support interviews, demos, sprint reviews, QA recordings,
+support sessions, discovery calls, and voice notes as task evidence. The first
+release should be an on-demand CLI/service workflow, not a real-time meeting
+bot.
 
-Every transcript evidence artifact should include:
+## Acceptance Criteria From GH-367
 
-- task id and actor;
-- workflow-local source artifact or approved evidence reference;
-- source hash;
-- duration and language when available;
-- engine/provider/model;
-- timestamp;
-- consent and retention notes when provided;
-- redaction policy;
-- acceptance-criteria mapping with timestamp ranges;
-- decisions, risks, defects, action items, unresolved questions, and lesson
-  candidates.
+- The skill activates on demand from transcription, audio, video, demo
+  recording, sprint review, interview, QA evidence, support call, or discovery
+  session signals.
+- Local/offline transcription is the default path when available.
+- External provider transcription requires explicit policy opt-in.
+- Provenance records source file path or artifact id, hash, duration, language,
+  engine/provider/model, timestamp, actor, task id, and consent/retention notes
+  when provided.
+- Outputs include human-readable Markdown and structured JSON; VTT/SRT are
+  available when timestamps are reliable.
+- The skill extracts speakers, timestamps, decisions, risks, action items,
+  acceptance-criteria candidates, defects, and lesson-learned candidates.
+- Sensitive values are redacted before persistence.
+- QA evidence can reference transcripts and timestamp ranges.
+- Failure modes are explicit and non-destructive.
+- Tests cover activation, policy gating, redaction, provenance, output formats,
+  artifact path safety, and degraded local/offline behavior.
+- Documentation explains local-first setup, privacy defaults, provider opt-in,
+  retention, and evidence usage.
 
-Preferred outputs are Markdown for human review and JSON for tooling. VTT and
-SRT are appropriate only when timestamp confidence is good enough for reviewers
-to navigate the media.
+## Architecture Decision
 
-## Failure Handling
+Status: proposed
 
-The skill should fail closed or produce degraded evidence for missing local
-engines, unsupported codecs, oversized media, unreadable paths, policy-blocked
-providers, redaction failures, partial transcripts, and missing regulated
-consent data.
+Decision: build a provider-neutral transcription pipeline with a local engine as
+the default adapter, an external-provider adapter family behind explicit policy
+gates, and a single evidence artifact contract shared by Markdown, JSON, VTT,
+and SRT exporters.
 
-## Example Task Signals
+Rationale:
+
+- The product need is governed evidence, not generic media processing.
+- Local-first execution minimizes default privacy and compliance risk.
+- Provider-neutral interfaces let the product add `whisper.cpp`,
+  `faster-whisper`, OpenAI, Google Cloud Speech-to-Text, or future adapters
+  without changing the evidence contract.
+- A single provenance envelope makes transcripts auditable across local and
+  external execution.
+- Redaction must happen before persisted summaries, handoffs, and evidence
+  links are written.
+
+Alternatives considered:
+
+| Option | Benefits | Costs / Risks | Decision |
+| --- | --- | --- | --- |
+| Local-only skill | Strong privacy posture, works offline after setup, lower vendor risk | Hardware variability, slower on large files, local engine install friction | Use as default path |
+| External-only provider | Better managed scaling and language/diarization options | Media leaves workspace, policy/compliance burden, cost controls required | Reject as default |
+| Provider-specific OpenAI or Google command | Fast MVP if one provider is already approved | Locks product contract to one vendor shape | Reject for core architecture |
+| Manual transcript attachment only | Minimal implementation | Does not satisfy evidence, provenance, or redaction requirements | Reject |
+
+## Scope
+
+In scope for the first implementation:
+
+- Skill activation metadata and on-demand command/service contract.
+- Workflow-local source validation for files and evidence artifact references.
+- Audio extraction and media metadata probing through a bounded media adapter.
+- Local transcription adapter contract with one supported local engine.
+- Explicit opt-in policy contract for external providers.
+- Transcript normalization, redaction, provenance, and artifact writing.
+- Markdown and JSON transcript evidence output.
+- VTT/SRT export when segment timestamps are available.
+- QA evidence mapping from acceptance criteria to timestamp ranges.
+
+Out of scope:
+
+- Real-time meeting attendance or live call capture.
+- Automatic external uploads.
+- Medical, legal, or financial interpretation as final advice.
+- Long-term media archive management beyond retention metadata.
+- Speaker identification as identity proof. Diarization is a best-effort label.
+
+## Proposed Component Boundaries
+
+```text
+Task / CLI / API request
+  -> Skill activation planner
+  -> Transcription request validator
+  -> Media probe and extraction adapter
+  -> Policy gate
+  -> Engine adapter
+  -> Transcript normalizer
+  -> Redaction service
+  -> Finding extractor
+  -> Evidence artifact writer
+```
+
+Recommended modules for implementation:
+
+| Boundary | Responsibility | Notes |
+| --- | --- | --- |
+| `transcription-request` domain | Narrow request, policy, output, and failure types | No provider-specific fields in public APIs except through typed adapter config |
+| `transcription-service` | Orchestrates validation, media probing, engine execution, redaction, extraction, and artifact writing | No shell string interpolation; adapters receive args arrays |
+| `media-probe-adapter` | Reads duration, codec, stream info, and extracts audio when needed | Wrap `ffprobe`/`ffmpeg` with `spawn`/`execFile` args arrays |
+| `transcription-engine-adapter` | Runs local or external provider and returns normalized segments | Provider-specific parsing stays here |
+| `transcript-redaction-service` | Applies secret, credential, configured PII, and regulated marker redaction | Must fail closed when redaction cannot run |
+| `transcript-evidence-writer` | Writes Markdown, JSON, VTT, SRT, and evidence metadata | Atomic writes; no raw unredacted transcript persistence |
+| `workflow-evidence-integration` | Links transcript artifact ids to task evidence and acceptance criteria | Keeps QA evidence references stable |
+
+## Local-First Stack Options
+
+Use `ffmpeg`/`ffprobe` as the media utility layer when present. The
+implementation should detect missing binaries and return a degraded evidence
+result rather than requiring installation during command execution.
+
+| Stack | Fit | Strengths | Constraints | Recommended use |
+| --- | --- | --- | --- | --- |
+| `whisper.cpp` | Local CPU/GPU transcription | Portable C/C++ runtime, offline operation, simple deployment model | Model download and hardware-dependent speed; diarization is not the core strength | Default MVP local engine candidate |
+| `faster-whisper` | Local or self-hosted accelerated Whisper inference | CTranslate2-backed performance, useful with GPU or optimized CPU | Python/runtime packaging and model cache management | Phase 2 local high-throughput adapter |
+| OS-native speech APIs | Local-ish desktop support where available | Low setup on specific platforms | Platform variance, privacy terms differ, not CI-friendly | Defer unless a product platform needs it |
+| Self-hosted transcription service | Team-managed private endpoint | Centralized resource control, can use GPUs | Becomes infrastructure and access-control work | Later enterprise option |
+
+Local setup principles:
+
+- Never auto-download models without explicit user action or config.
+- Store model cache locations in typed configuration.
+- Record model name, model path or digest when available, and runtime version.
+- Enforce default size and duration limits before invoking engines.
+- Keep temp audio files inside a workflow-owned temp directory and delete them
+  after successful artifact generation unless retention policy says otherwise.
+
+## External Provider Opt-In
+
+External transcription is allowed only when all of these are true:
+
+- The task or workspace policy explicitly enables the provider.
+- The source artifact is classified as allowed for external processing.
+- Required consent and retention notes are present for regulated or user
+  research recordings.
+- Secrets and obvious credentials are screened before upload where technically
+  possible.
+- Cost, size, duration, and rate-limit budgets are configured.
+- The evidence artifact records provider, model, endpoint family, request id or
+  equivalent correlation id, timestamp, actor, and policy id.
+
+Provider candidates:
+
+| Provider family | Fit | Required controls |
+| --- | --- | --- |
+| OpenAI Audio transcription | Optional hosted provider for approved workspaces | Server-side API key only, HTTPS, request/correlation id logging, pinned model config, no client-side key exposure |
+| Google Cloud Speech-to-Text | Optional hosted provider for enterprise Google Cloud tenants | IAM, regional endpoint review, audit logging, quota/budget controls, encryption and retention review |
+| Other providers | Future adapters | Same policy gate and provenance contract before enablement |
+
+External-provider adapters must not be referenced from product logic directly.
+The service should receive a `TranscriptionEngineAdapter` selected by policy and
+configuration.
+
+## Privacy, Security, And Compliance Controls
+
+Data classification:
+
+| Data | Default classification | Handling |
+| --- | --- | --- |
+| Source audio/video | Sensitive evidence | Workflow-local only by default; external provider requires opt-in |
+| Raw transcript before redaction | Sensitive derived data | In memory or temp-only; do not persist unless explicitly configured for debugging in a secure path |
+| Redacted transcript | Workflow evidence | Persist as Markdown/JSON under evidence output path |
+| Metadata/provenance | Audit data | Persist with artifact; avoid leaking local absolute paths in user-facing output when not needed |
+| Consent/retention notes | Compliance metadata | Required for regulated/user research contexts before release |
+
+Security requirements:
+
+- Validate paths are inside the workspace or approved evidence directories.
+- Reject traversal, symlinks that resolve outside allowed roots, unreadable
+  files, and unsupported artifact references.
+- Use `spawn` or `execFile` with args arrays for `ffmpeg`, `ffprobe`, local
+  engines, and helper commands.
+- Validate external URLs use `https://` and configured allowlisted hosts.
+- Load provider secrets only from server-side environment or secret manager
+  integration; never write them to transcript artifacts or logs.
+- Redact before persistence and before summary generation.
+- Fail closed on redaction failure, provider policy denial, missing consent
+  where required, or unsupported file path.
+- Cap file size, duration, segment count, output size, and provider cost.
+- Store failure reports without copying raw transcript or media bytes.
+
+Compliance requirements:
+
+- Record consent status as one of `not_required`, `provided`, `missing`, or
+  `unknown`.
+- Record retention class and delete/review date when provided.
+- Mark regulated domain signals: health, finance, legal, government id,
+  children/minors, biometric voice data, and customer support PII.
+- Require human review before using transcript-derived medical, legal,
+  financial, hiring, or support-account decisions as final determinations.
+- Keep tenant policy explicit; do not infer permission from provider credentials
+  alone.
+
+## Evidence Artifact Contract
+
+Each transcript run should write one artifact set with a stable id:
+
+```text
+.agent-workflow/evidence/transcripts/<task-id>/<artifact-id>.md
+.agent-workflow/evidence/transcripts/<task-id>/<artifact-id>.json
+.agent-workflow/evidence/transcripts/<task-id>/<artifact-id>.vtt
+.agent-workflow/evidence/transcripts/<task-id>/<artifact-id>.srt
+```
+
+VTT/SRT files are optional and should be omitted when timestamp confidence is
+too low.
+
+Minimum JSON shape:
+
+```json
+{
+  "schemaVersion": 1,
+  "taskId": "GH-367-TRANSCRIPTION-SKILL-SPIKE",
+  "artifactId": "transcript_20260522_000001",
+  "source": {
+    "kind": "workflow-file",
+    "path": ".agent-workflow/evidence/demo.mp4",
+    "sha256": "hex",
+    "durationMs": 120000,
+    "mimeType": "video/mp4",
+    "language": "en"
+  },
+  "engine": {
+    "executionMode": "local",
+    "provider": "whisper.cpp",
+    "model": "base.en",
+    "version": "detected-version"
+  },
+  "policy": {
+    "policyId": "workspace-default",
+    "externalProviderAllowed": false,
+    "redactionPolicyId": "default",
+    "consentStatus": "provided",
+    "retentionClass": "task-evidence"
+  },
+  "provenance": {
+    "actor": "qa",
+    "generatedAt": "2026-05-22T00:00:00.000Z",
+    "command": "orchestra transcript run",
+    "requestId": "local-run-id"
+  },
+  "segments": [
+    {
+      "startMs": 1000,
+      "endMs": 3500,
+      "speaker": "speaker_1",
+      "text": "Redacted transcript segment.",
+      "confidence": 0.92
+    }
+  ],
+  "findings": {
+    "decisions": [],
+    "risks": [],
+    "actionItems": [],
+    "acceptanceCriteriaCandidates": [],
+    "defects": [],
+    "lessonCandidates": [],
+    "unresolvedQuestions": []
+  },
+  "redactions": {
+    "applied": true,
+    "countsByType": {
+      "secret": 0,
+      "email": 2,
+      "phone": 1,
+      "regulatedMarker": 0
+    }
+  },
+  "quality": {
+    "isPartial": false,
+    "timestampConfidence": "high",
+    "warnings": []
+  }
+}
+```
+
+Markdown report sections:
+
+- Summary and provenance.
+- Source, policy, consent, retention, and redaction status.
+- Acceptance-criteria mapping table with timestamp ranges.
+- Decisions, risks, defects, action items, lesson candidates, and unresolved
+  questions.
+- Gaps and degraded-mode warnings.
+
+## Failure Model
+
+All failures must be non-destructive and evidence-friendly.
+
+| Failure | Expected behavior |
+| --- | --- |
+| Missing `ffmpeg`/`ffprobe` | Return degraded result with setup guidance; do not mutate source |
+| Missing local engine | Return policy-safe degraded evidence; do not fall back to external provider automatically |
+| Unsupported codec | Record media probe failure and suggested conversion path |
+| Oversized file or duration | Stop before engine invocation and record configured limit |
+| Path outside allowed roots | Block as security failure |
+| Provider not opted in | Block external execution and record policy id |
+| Provider timeout/rate limit | Record partial/deferred result with retry-safe metadata |
+| Redaction failure | Fail closed; do not persist transcript text |
+| Low timestamp confidence | Write Markdown/JSON with warning; omit VTT/SRT |
+| Partial transcript | Mark `quality.isPartial=true` and require QA review before release |
+| Missing consent in regulated context | Block release approval or require PO risk acceptance |
+
+## QA Evidence And Test Strategy
+
+QA must map every GH-367 acceptance criterion to an observable check. The first
+implementation should prefer deterministic fixtures: tiny generated WAV/video
+fixtures, mocked engine adapters, and provider contract fakes.
+
+| Acceptance area | Test type | Fixture/setup | Expected evidence |
+| --- | --- | --- | --- |
+| Skill activation | Unit/CLI | Task text with transcription/audio/video signals and unrelated control task | Skill selected only for matching signals |
+| Local default | Unit/service | Local engine available; no provider opt-in | Adapter selection uses local engine |
+| Provider opt-in | Unit/contract | Provider configured with policy denied/allowed variants | Denied blocks; allowed records provider provenance |
+| Provenance | Unit/snapshot | Mock media metadata and engine result | JSON includes source hash, duration, language, model, actor, task id, timestamp |
+| Markdown/JSON output | Unit/golden file | Mock segments and findings | Markdown and JSON match schema; no unredacted secret |
+| VTT/SRT output | Unit/golden file | Timestamped and untimestamped segment variants | Subtitles emitted only with adequate timestamps |
+| Finding extraction | Unit | Transcript with decisions, risks, action items, defects | Structured findings populate expected arrays |
+| Redaction | Unit/security | Secrets, tokens, emails, phones, regulated markers | Redacted before persistence; counts recorded |
+| Path safety | Unit/security | Traversal, symlink escape, valid evidence path | Unsafe paths blocked |
+| Missing tools | Unit/integration | `ffmpeg` or engine unavailable | Degraded evidence result, no provider fallback |
+| Oversized media | Unit | File size/duration over limits | Engine not invoked; clear failure |
+| QA timestamp references | Integration | Transcript artifact linked to AC table | Evidence can reference timestamp ranges |
+
+Recommended commands once implemented:
 
 ```bash
-orchestra task add \
-  --id QA-TRANSCRIPT-001 \
-  --title "Transcribe sprint review demo recording into QA evidence" \
-  --owner qa \
-  --paths ".agent-workflow/evidence/demo.mp4,docs/evidence/transcripts" \
-  --acceptance "recording transcript maps acceptance criteria to timestamps" \
-  --risks "privacy,security,governance,release"
-
-orchestra skills plan --task QA-TRANSCRIPT-001
+node --test test/transcription-*.test.js
+npm run build
+npm run precommit
 ```
+
+Manual QA for the first release:
+
+- Run a small local audio fixture through the command.
+- Verify generated Markdown and JSON artifact paths.
+- Verify no raw unredacted transcript is persisted when fixture contains
+  secret-like and PII-like text.
+- Verify provider-denied policy cannot upload.
+- Verify provider-allowed policy records the opt-in decision and correlation
+  metadata.
+
+## Developer Implementation Slices
+
+1. `GH-367A` - Skill activation and request contract
+   - Add skill manifest/catalog wiring if needed.
+   - Define request, policy, output, segment, finding, and failure types.
+   - Tests: activation signals, narrow public types, invalid request handling.
+
+2. `GH-367B` - Media validation and local probe adapter
+   - Validate workspace/evidence paths.
+   - Add `ffprobe` metadata probe and `ffmpeg` audio extraction wrapper using
+     args arrays.
+   - Tests: path traversal, symlink escape, missing binary, unsupported codec,
+     size/duration limits.
+
+3. `GH-367C` - Local engine adapter MVP
+   - Add one local engine adapter, preferably `whisper.cpp`, behind the generic
+     `TranscriptionEngineAdapter`.
+   - Normalize output into segments.
+   - Tests: adapter command construction, missing engine, partial transcript,
+     timestamp confidence.
+
+4. `GH-367D` - Evidence writer and provenance artifacts
+   - Write Markdown/JSON and optional VTT/SRT from normalized transcript.
+   - Add atomic artifact writes and evidence registration.
+   - Tests: schema, golden Markdown, subtitle gating, artifact id stability.
+
+5. `GH-367E` - Redaction and privacy policy gates
+   - Add redaction pipeline before persistence.
+   - Add provider opt-in policy and consent/retention checks.
+   - Tests: secrets, configured PII, regulated markers, redaction failure,
+     missing consent, provider denial.
+
+6. `GH-367F` - External provider adapter behind opt-in
+   - Add first hosted provider adapter only after policy gates are tested.
+   - Record request/correlation id, provider/model, region/endpoint family when
+     available, and cost/duration guardrail metadata.
+   - Tests: mocked provider success, timeout, rate limit, malformed response,
+     no secret leakage.
+
+7. `GH-367G` - QA evidence mapping and docs
+   - Add acceptance criteria timestamp mapping support.
+   - Document setup, local engine install expectations, provider opt-in,
+     retention, and troubleshooting.
+   - Tests: evidence report links transcript timestamp ranges to AC rows.
+
+## Risks And Mitigations
+
+| Risk | Severity | Mitigation |
+| --- | --- | --- |
+| Secret or PII leakage from recordings | High | Local default, explicit external opt-in, redaction before persistence, security tests |
+| Raw transcript persistence before redaction | High | In-memory/temp-only handling; fail closed on redaction errors |
+| Provider credentials imply accidental opt-in | High | Separate credentials from policy; require explicit provider policy id |
+| Transcript hallucination or speaker misattribution | Medium | Mark confidence/quality, require human review, avoid identity claims |
+| Large media files exhaust local resources | Medium | Size/duration/segment limits and preflight probing |
+| Local engine availability differs by machine/CI | Medium | Degraded failure model and mocked deterministic tests |
+| Codec variance causes flaky tests | Medium | Tiny generated fixtures and mocked media probe for most tests |
+| Regulated retention/consent varies by tenant | High | Required metadata fields and release blocking for missing regulated consent |
+| External provider terms or model behavior changes | Medium | Adapter isolation, pinned model config, provider-specific docs, eval fixtures |
+
+## Open Questions
+
+- Where should transcript artifacts live long term: `.agent-workflow/evidence`
+  only, or a configurable workspace evidence root?
+- Should raw transcripts ever be retained for debugging, and if so under what
+  encrypted local policy?
+- Which local engine should be the MVP implementation target for supported
+  platforms?
+- Which provider, if any, has an approved tenant policy for the first hosted
+  adapter?
+- Should diarization be included in MVP or treated as a best-effort optional
+  post-processor?
+
+## Recommended Next Stories
+
+- `GH-367A-SKILL-CONTRACT`: Define transcription request/output/policy types and
+  skill activation behavior.
+- `GH-367B-MEDIA-PREFLIGHT`: Implement workflow-local media validation,
+  `ffprobe` metadata extraction, and safe `ffmpeg` audio extraction.
+- `GH-367C-LOCAL-ENGINE`: Add the first local transcription adapter with
+  degraded-mode behavior.
+- `GH-367D-EVIDENCE-ARTIFACTS`: Generate Markdown/JSON/VTT/SRT transcript
+  artifacts with provenance.
+- `GH-367E-PRIVACY-REDACTION`: Implement redaction, consent, retention, and
+  provider policy gates.
+- `GH-367F-PROVIDER-ADAPTER`: Add one external provider adapter behind explicit
+  opt-in and mocked contract tests.
+- `GH-367G-QA-EVIDENCE`: Add AC-to-timestamp evidence mapping and QA report
+  workflow.
+
+## Reference Notes
+
+- `whisper.cpp` is an active local C/C++ port of Whisper suitable for offline
+  transcription experiments.
+- `faster-whisper` uses CTranslate2 and is a candidate for faster local or
+  self-hosted inference.
+- OpenAI and Google Cloud Speech-to-Text are viable hosted transcription
+  adapter families only behind explicit workspace policy opt-in.

package/docs/autonomous-workflow.md

@@ -25,6 +25,44 @@ The run state, gate artifacts, handoffs, evidence, reviews, decisions, and
 clarifications are persisted under `.agent-workflow/` so the delivery story can
 be audited after the fact.
 
+## Task-Scoped Roles
+
+Tasks can declare the roles that are required or optional for the workflow. When
+a task is created with only an owner role, Open Orchestra treats that owner as
+the implicit required role instead of falling back to the default delivery
+workflow.
+
+```bash
+orchestra task add --id BUG-001 --title "Fix CLI bug" --owner developer
+orchestra workflow run --task BUG-001 --gates phase
+```
+
+Use explicit roles when the task needs a broader lifecycle:
+
+```bash
+orchestra task add --id STORY-001 --title "Ship user-facing workflow" \
+  --owner product_owner --required-roles product_owner,architect,developer,qa,release_manager
+```
+
+This keeps small developer or QA tasks scoped while still allowing full
+PO-to-release delivery for stories that need it.
+
+## Workspace Isolation
+
+Workflow commands that operate on run state support `--target-dir` so callers
+can launch work from another directory without writing `.agent-workflow/` state
+to the wrong repo.
+
+```bash
+orchestra workflow run --task STORY-001 --target-dir /path/to/project
+orchestra workflow list --target-dir /path/to/project
+orchestra workflow gate-approve --run <run-id> --target-dir /path/to/project
+```
+
+Use `--target-dir` for temporary E2E projects, editor integrations, local web
+console actions, and any parent process that coordinates more than one
+workspace.
+
 ## Phase Graph
 
 ```