@jterrats/open-orchestra 1.0.14 → 1.0.16
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +7 -4
- package/CHANGELOG.md +26 -0
- package/CLAUDE.md +2 -2
- package/dist/automation-evidence.d.ts +2 -23
- package/dist/automation-evidence.js +1 -218
- package/dist/automation-evidence.js.map +1 -1
- package/dist/command-manifest.js +12 -3
- package/dist/command-manifest.js.map +1 -1
- package/dist/command-routes-integrations.js +2 -1
- package/dist/command-routes-integrations.js.map +1 -1
- package/dist/command-routes.js +11 -1
- package/dist/command-routes.js.map +1 -1
- package/dist/commands.d.ts +5 -3
- package/dist/commands.js +9 -3
- package/dist/commands.js.map +1 -1
- package/dist/context-index-commands.d.ts +2 -0
- package/dist/context-index-commands.js +51 -1
- package/dist/context-index-commands.js.map +1 -1
- package/dist/context-index-service.d.ts +2 -1
- package/dist/context-index-service.js +3 -1
- package/dist/context-index-service.js.map +1 -1
- package/dist/context-index-signals.d.ts +2 -0
- package/dist/context-index-signals.js +102 -0
- package/dist/context-index-signals.js.map +1 -0
- package/dist/context-pack-service.d.ts +14 -0
- package/dist/context-pack-service.js +153 -0
- package/dist/context-pack-service.js.map +1 -0
- package/dist/context-search-service.d.ts +10 -0
- package/dist/context-search-service.js +142 -0
- package/dist/context-search-service.js.map +1 -0
- package/dist/context-vault-commands.d.ts +5 -0
- package/dist/context-vault-commands.js +79 -0
- package/dist/context-vault-commands.js.map +1 -0
- package/dist/context-vault-file-metadata.d.ts +3 -0
- package/dist/context-vault-file-metadata.js +25 -0
- package/dist/context-vault-file-metadata.js.map +1 -0
- package/dist/context-vault-model.d.ts +79 -0
- package/dist/context-vault-model.js +2 -0
- package/dist/context-vault-model.js.map +1 -0
- package/dist/context-vault-redaction.d.ts +5 -0
- package/dist/context-vault-redaction.js +22 -0
- package/dist/context-vault-redaction.js.map +1 -0
- package/dist/context-vault-renderer.d.ts +3 -0
- package/dist/context-vault-renderer.js +30 -0
- package/dist/context-vault-renderer.js.map +1 -0
- package/dist/context-vault-service.d.ts +17 -0
- package/dist/context-vault-service.js +183 -0
- package/dist/context-vault-service.js.map +1 -0
- package/dist/delivery-commands.d.ts +1 -0
- package/dist/delivery-commands.js +19 -1
- package/dist/delivery-commands.js.map +1 -1
- package/dist/diagram-schema.d.ts +2 -0
- package/dist/diagram-schema.js +148 -0
- package/dist/diagram-schema.js.map +1 -0
- package/dist/diagrams/geometry.d.ts +12 -0
- package/dist/diagrams/geometry.js +69 -0
- package/dist/diagrams/geometry.js.map +1 -0
- package/dist/diagrams/iconify.d.ts +22 -0
- package/dist/diagrams/iconify.js +42 -0
- package/dist/diagrams/iconify.js.map +1 -0
- package/dist/diagrams/index.d.ts +8 -0
- package/dist/diagrams/index.js +8 -0
- package/dist/diagrams/index.js.map +1 -0
- package/dist/diagrams/layout.d.ts +2 -0
- package/dist/diagrams/layout.js +142 -0
- package/dist/diagrams/layout.js.map +1 -0
- package/dist/diagrams/model.d.ts +94 -0
- package/dist/diagrams/model.js +2 -0
- package/dist/diagrams/model.js.map +1 -0
- package/dist/diagrams/pipeline.d.ts +11 -0
- package/dist/diagrams/pipeline.js +92 -0
- package/dist/diagrams/pipeline.js.map +1 -0
- package/dist/diagrams/render-svg.d.ts +7 -0
- package/dist/diagrams/render-svg.js +66 -0
- package/dist/diagrams/render-svg.js.map +1 -0
- package/dist/diagrams/validate.d.ts +5 -0
- package/dist/diagrams/validate.js +120 -0
- package/dist/diagrams/validate.js.map +1 -0
- package/dist/evidence-compaction-model.d.ts +62 -0
- package/dist/evidence-compaction-model.js +2 -0
- package/dist/evidence-compaction-model.js.map +1 -0
- package/dist/evidence-compaction-renderer.d.ts +2 -0
- package/dist/evidence-compaction-renderer.js +35 -0
- package/dist/evidence-compaction-renderer.js.map +1 -0
- package/dist/evidence-compaction-service.d.ts +11 -0
- package/dist/evidence-compaction-service.js +94 -0
- package/dist/evidence-compaction-service.js.map +1 -0
- package/dist/evidence-compaction-summary.d.ts +4 -0
- package/dist/evidence-compaction-summary.js +160 -0
- package/dist/evidence-compaction-summary.js.map +1 -0
- package/dist/handoff-acceptance-metadata.d.ts +6 -0
- package/dist/handoff-acceptance-metadata.js +8 -0
- package/dist/handoff-acceptance-metadata.js.map +1 -0
- package/dist/knowledge-base.js +7 -1
- package/dist/knowledge-base.js.map +1 -1
- package/dist/model-providers.d.ts +3 -1
- package/dist/model-providers.js +68 -11
- package/dist/model-providers.js.map +1 -1
- package/dist/performance-benchmark.d.ts +45 -0
- package/dist/performance-benchmark.js +134 -0
- package/dist/performance-benchmark.js.map +1 -0
- package/dist/phase-executor.js +24 -6
- package/dist/phase-executor.js.map +1 -1
- package/dist/planning-commands.js +23 -1
- package/dist/planning-commands.js.map +1 -1
- package/dist/provider-agent-wrapper.d.ts +12 -0
- package/dist/provider-agent-wrapper.js +264 -0
- package/dist/provider-agent-wrapper.js.map +1 -0
- package/dist/qa-commands.d.ts +1 -0
- package/dist/qa-commands.js +26 -0
- package/dist/qa-commands.js.map +1 -1
- package/dist/qa-evidence-surface-rules.d.ts +5 -0
- package/dist/qa-evidence-surface-rules.js +238 -0
- package/dist/qa-evidence-surface-rules.js.map +1 -0
- package/dist/qa-evidence-validation.d.ts +23 -0
- package/dist/qa-evidence-validation.js +78 -0
- package/dist/qa-evidence-validation.js.map +1 -0
- package/dist/quality-contracts.js +21 -6
- package/dist/quality-contracts.js.map +1 -1
- package/dist/release-candidate.d.ts +4 -0
- package/dist/release-candidate.js +54 -0
- package/dist/release-candidate.js.map +1 -1
- package/dist/release-commands.js +8 -0
- package/dist/release-commands.js.map +1 -1
- package/dist/release-readiness.d.ts +8 -0
- package/dist/release-readiness.js +59 -4
- package/dist/release-readiness.js.map +1 -1
- package/dist/release-reviewer-selection.d.ts +13 -0
- package/dist/release-reviewer-selection.js +135 -0
- package/dist/release-reviewer-selection.js.map +1 -0
- package/dist/report-index.d.ts +3 -0
- package/dist/report-index.js +3 -0
- package/dist/report-index.js.map +1 -0
- package/dist/report-model.d.ts +22 -0
- package/dist/report-model.js +2 -0
- package/dist/report-model.js.map +1 -0
- package/dist/report-render-markdown.d.ts +1 -0
- package/dist/report-render-markdown.js +40 -0
- package/dist/report-render-markdown.js.map +1 -0
- package/dist/report-schema.d.ts +2 -0
- package/dist/report-schema.js +109 -0
- package/dist/report-schema.js.map +1 -0
- package/dist/rule-catalog.d.ts +33 -0
- package/dist/rule-catalog.js +215 -0
- package/dist/rule-catalog.js.map +1 -0
- package/dist/runtime-adapters.js +4 -1
- package/dist/runtime-adapters.js.map +1 -1
- package/dist/runtime-bootstrap.js +11 -0
- package/dist/runtime-bootstrap.js.map +1 -1
- package/dist/runtime-child-prompt.d.ts +14 -0
- package/dist/runtime-child-prompt.js +49 -0
- package/dist/runtime-child-prompt.js.map +1 -0
- package/dist/runtime-commands.js +7 -2
- package/dist/runtime-commands.js.map +1 -1
- package/dist/runtime-context-manifest.d.ts +15 -1
- package/dist/runtime-context-manifest.js +24 -25
- package/dist/runtime-context-manifest.js.map +1 -1
- package/dist/runtime-context-pack-reference.d.ts +10 -0
- package/dist/runtime-context-pack-reference.js +37 -0
- package/dist/runtime-context-pack-reference.js.map +1 -0
- package/dist/runtime-execution-renderer.js +6 -0
- package/dist/runtime-execution-renderer.js.map +1 -1
- package/dist/runtime-execution.js +8 -4
- package/dist/runtime-execution.js.map +1 -1
- package/dist/runtime-lifecycle-notifications.d.ts +28 -0
- package/dist/runtime-lifecycle-notifications.js +110 -0
- package/dist/runtime-lifecycle-notifications.js.map +1 -0
- package/dist/runtime-lifecycle-watch-adapters.js +22 -3
- package/dist/runtime-lifecycle-watch-adapters.js.map +1 -1
- package/dist/runtime-lifecycle-watch.d.ts +1 -1
- package/dist/runtime-lifecycle-watch.js +55 -27
- package/dist/runtime-lifecycle-watch.js.map +1 -1
- package/dist/runtime-notification-commands.d.ts +2 -0
- package/dist/runtime-notification-commands.js +31 -0
- package/dist/runtime-notification-commands.js.map +1 -0
- package/dist/runtime-parent-action-dispatch.js +9 -0
- package/dist/runtime-parent-action-dispatch.js.map +1 -1
- package/dist/runtime-parent-actions.d.ts +2 -1
- package/dist/runtime-parent-actions.js +17 -6
- package/dist/runtime-parent-actions.js.map +1 -1
- package/dist/runtime-spawn-bridge.js +54 -41
- package/dist/runtime-spawn-bridge.js.map +1 -1
- package/dist/structured-output-validation.d.ts +9 -0
- package/dist/structured-output-validation.js +20 -0
- package/dist/structured-output-validation.js.map +1 -0
- package/dist/task-graph-commands.js +1 -1
- package/dist/task-graph-commands.js.map +1 -1
- package/dist/types/context-index.d.ts +57 -0
- package/dist/types/model-config.d.ts +93 -0
- package/dist/types/runtime.d.ts +10 -1
- package/dist/types.d.ts +4 -2
- package/dist/types.js.map +1 -1
- package/dist/web-api-read-routes.js +2 -0
- package/dist/web-api-read-routes.js.map +1 -1
- package/dist/web-api.js +45 -22
- package/dist/web-api.js.map +1 -1
- package/dist/web-console/assets/{index-jxCY5eEc.css → index-BHs7OIv8.css} +1 -1
- package/dist/web-console/assets/index-BJuVTqfQ.js +11 -0
- package/dist/web-console/index.html +2 -2
- package/dist/workflow-evidence-service.js +18 -3
- package/dist/workflow-evidence-service.js.map +1 -1
- package/dist/workflow-gates.js +1 -13
- package/dist/workflow-gates.js.map +1 -1
- package/dist/workflow-handoff-assessment.js +35 -2
- package/dist/workflow-handoff-assessment.js.map +1 -1
- package/dist/workflow-handoff-gates.d.ts +2 -0
- package/dist/workflow-handoff-gates.js +28 -0
- package/dist/workflow-handoff-gates.js.map +1 -0
- package/dist/workflow-run-commands.js +78 -1
- package/dist/workflow-run-commands.js.map +1 -1
- package/dist/workflow-services.d.ts +1 -0
- package/dist/workflow-services.js +132 -151
- package/dist/workflow-services.js.map +1 -1
- package/dist/workspace-claude-settings.d.ts +27 -0
- package/dist/workspace-claude-settings.js +54 -0
- package/dist/workspace-claude-settings.js.map +1 -0
- package/dist/workspace-runtime-bootstrap.js +5 -0
- package/dist/workspace-runtime-bootstrap.js.map +1 -1
- package/dist/workspace-selection.d.ts +12 -0
- package/dist/workspace-selection.js +25 -0
- package/dist/workspace-selection.js.map +1 -0
- package/docs/autonomous-workflow.md +3 -3
- package/docs/context-index-packs.md +45 -2
- package/docs/context-vault.md +34 -11
- package/docs/diagrams/deterministic-pipeline/README.md +102 -0
- package/docs/e2e-test-batteries.md +3 -3
- package/docs/evidence-compaction.md +25 -0
- package/docs/reports/context-pack-benchmark-gh-452.json +119 -0
- package/docs/reports/context-pack-benchmark-gh-452.md +32 -0
- package/docs/reports/gh-428-test-coverage-context-review-20260522.md +75 -0
- package/docs/rule-loading-strategy.md +37 -0
- package/docs/runtime-adapters.md +138 -27
- package/docs/security-env-vars.md +41 -0
- package/docs/sonar-quality-gates.md +23 -1
- package/package.json +4 -2
- package/rules/development/semantic-code.md +28 -0
- package/rules/diagram-quality.mdc +2 -0
- package/dist/web-console/assets/index-DA8Fs4r7.js +0 -11
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
# Deterministic Diagram Pipeline
|
|
2
|
+
|
|
3
|
+
Task: GH-462-DETERMINISTIC-DIAGRAM-PIPELINE
|
|
4
|
+
Status: first implementation slice
|
|
5
|
+
|
|
6
|
+
## Scope
|
|
7
|
+
|
|
8
|
+
This slice introduces a local TypeScript pipeline for source-free semantic
|
|
9
|
+
diagrams:
|
|
10
|
+
|
|
11
|
+
1. `DiagramModel` captures semantic intent: nodes, groups, connectors, labels,
|
|
12
|
+
and Iconify icon references.
|
|
13
|
+
2. `layoutDiagram()` produces deterministic geometry from model order and group
|
|
14
|
+
membership.
|
|
15
|
+
3. `renderDiagramSvg()` emits a minimal SVG artifact from the model and layout.
|
|
16
|
+
4. `validateDiagramArtifact()` checks text fit, canvas/container containment,
|
|
17
|
+
connector endpoints, connector-node overlaps, connector labels covering
|
|
18
|
+
other lines, and unnecessary bends where the path is already straight.
|
|
19
|
+
5. `generateDeterministicDiagram()` ties model, layout, rendering, and
|
|
20
|
+
validation together for a single pass. It first validates the runtime
|
|
21
|
+
payload against the structured diagram schema and fails with field-level
|
|
22
|
+
errors before any layout or SVG rendering occurs.
|
|
23
|
+
6. `runDeterministicDiagramPipeline()` adds bounded iteration. It renders the
|
|
24
|
+
first pass, applies deterministic text-fit repair when possible, regenerates
|
|
25
|
+
the artifact, and retains only the final artifact unless
|
|
26
|
+
`retainIterations=true` is requested for debug evidence.
|
|
27
|
+
|
|
28
|
+
The current slice is intentionally API-level. CLI routing is still pending so
|
|
29
|
+
this does not widen `command-routes*` or `tool-commands` ownership in the same
|
|
30
|
+
change. Consumers can call `runDeterministicDiagramPipeline()` directly to get a
|
|
31
|
+
stable final SVG plus optional retained iteration artifacts.
|
|
32
|
+
|
|
33
|
+
## Structured Output Contracts
|
|
34
|
+
|
|
35
|
+
Agents that generate deterministic artifacts must return structured JSON-like
|
|
36
|
+
payloads, not prose instructions for renderers to interpret.
|
|
37
|
+
|
|
38
|
+
Diagram payloads must satisfy `DiagramModel`:
|
|
39
|
+
|
|
40
|
+
- `id`, `title`, and `direction` are required.
|
|
41
|
+
- `direction` must be `right` or `down`.
|
|
42
|
+
- `nodes` must be a non-empty array. Each node needs `id`, `kind`, and
|
|
43
|
+
`text.label`; `kind` must be one of `actor`, `system`, `service`, `database`,
|
|
44
|
+
`queue`, or `boundary`.
|
|
45
|
+
- `connectors` must be an array. Each connector needs `id`, `from`, `to`, and
|
|
46
|
+
`kind`; `from` and `to` must reference known node ids; `kind` must be one of
|
|
47
|
+
`sync`, `async`, `data`, or `control`.
|
|
48
|
+
- Optional `groups` define labeled containers. Unused groups are allowed so
|
|
49
|
+
generated drafts can keep future grouping intent, but duplicate group ids are
|
|
50
|
+
rejected.
|
|
51
|
+
|
|
52
|
+
Report payloads use `ReportDocument` through `renderReportMarkdown()`:
|
|
53
|
+
|
|
54
|
+
- `id`, `title`, `summary`, and a non-empty `sections` array are required.
|
|
55
|
+
- Section `kind` must be one of `summary`, `findings`, `decisions`, `risks`,
|
|
56
|
+
`evidence`, or `nextSteps`.
|
|
57
|
+
- Section items require `id` and `text`; optional severity must be one of
|
|
58
|
+
`info`, `low`, `medium`, `high`, or `critical`.
|
|
59
|
+
|
|
60
|
+
Validation errors include JSON-path-like locations, for example
|
|
61
|
+
`$.connectors[0].to: references unknown node "api-gateway"`. Agent prompts
|
|
62
|
+
should pass those messages back to the generating role unchanged so the payload
|
|
63
|
+
can be corrected without guessing which field failed.
|
|
64
|
+
|
|
65
|
+
## Icon Policy
|
|
66
|
+
|
|
67
|
+
Diagram nodes reference icons by semantic purpose and Iconify id. Rendering
|
|
68
|
+
resolves the SVG body from an injected cache. Tests use the in-memory cache, so
|
|
69
|
+
the pipeline does not require network access to prove deterministic rendering.
|
|
70
|
+
|
|
71
|
+
Future slices can add a controlled cache warmer for Iconify sets. That command
|
|
72
|
+
should pin source metadata, record cache evidence, and fail closed when icons are
|
|
73
|
+
missing instead of fetching during tests.
|
|
74
|
+
|
|
75
|
+
## Target Guidance
|
|
76
|
+
|
|
77
|
+
- Use Mermaid for markdown-native semantic diagrams and fast syntax validation.
|
|
78
|
+
- Use this deterministic SVG pipeline when a generated diagram needs stable
|
|
79
|
+
geometry, validation findings, and repeatable rendering without relying on LLM
|
|
80
|
+
absolute coordinates.
|
|
81
|
+
- Use draw.io XML or Lucid when the acceptance target is editable,
|
|
82
|
+
pixel-sensitive recreation, manual stakeholder editing, or exact connector
|
|
83
|
+
anchors beyond this pipeline's current router.
|
|
84
|
+
- Use LLM prompt generation for semantic drafting, copy, and model suggestions;
|
|
85
|
+
do not treat prompt-selected coordinates as the source of truth.
|
|
86
|
+
|
|
87
|
+
## Artifact Retention
|
|
88
|
+
|
|
89
|
+
Default generation keeps the final SVG artifact and reports how many internal
|
|
90
|
+
iterations were discarded. Debug generation can request `retainIterations=true`
|
|
91
|
+
to keep each intermediate artifact and its findings. This keeps normal
|
|
92
|
+
deliverables clean while allowing QA or architecture review to inspect how the
|
|
93
|
+
pipeline corrected text fit and regenerated the diagram.
|
|
94
|
+
|
|
95
|
+
## Known Gaps
|
|
96
|
+
|
|
97
|
+
- Connector routing is simple orthogonal routing, not a full obstacle-avoidance
|
|
98
|
+
solver.
|
|
99
|
+
- Text measurement is deterministic approximation based on character and line
|
|
100
|
+
counts, not browser font metrics.
|
|
101
|
+
- Iteration artifact retention is available through API, but not exposed through
|
|
102
|
+
CLI yet.
|
|
@@ -36,11 +36,11 @@ entry points a user or CI runner actually executes.
|
|
|
36
36
|
|
|
37
37
|
| Battery | Scope | Command | Minimum Assertions | Evidence |
|
|
38
38
|
| ------------------------------ | --------------------------------------------------------------------- | --------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------ |
|
|
39
|
-
| Multi-squad runtime | Parallel squad delegation with queue and threshold policy | `
|
|
39
|
+
| Multi-squad runtime | Parallel squad delegation with queue and threshold policy | `npm run test:e2e:runtime` | independent sessions, non-blocking parent, queued sessions do not fall back to parent, completion order reconciles | JSON output, lifecycle events |
|
|
40
40
|
| Acceptance evidence | CLI, API, browser, and deferred integration evidence | `node --test e2e/acceptance-evidence.test.js` | evidence maps to named acceptance criteria, deferred external validation requires owner and rationale | evidence artifacts |
|
|
41
41
|
| Recovery and repair | Interrupted runs, stale locks, failed provider phases | `node --test e2e/recovery-cli.test.js` plus browser recovery coverage | recovery detects issue, repair requires confirmation, repaired state is observable | JSON output, before/after state |
|
|
42
42
|
| Docs/site content source | Site content generated from docs and manifest | `npm run site:build && npm run test:e2e -- --grep docs` | docs render as human-friendly catalog, no markdown-only dead ends, search works | Playwright report |
|
|
43
|
-
| Security-sensitive operations | File paths, shell execution, web writes, secrets, telemetry redaction | `
|
|
43
|
+
| Security-sensitive operations | File paths, shell execution, web writes, secrets, telemetry redaction | `npm run test:e2e:security` | path traversal blocked, unsafe writes rejected, secret-like data redacted, no raw stack traces | command/API evidence |
|
|
44
44
|
| Ollama provider-backed runtime | Local OpenAI-compatible Ollama provider route in a `/tmp` workspace | `npm run test:e2e:runtime:ollama` | `model connect --provider ollama`, provider-backed developer phase, OpenAI-compatible request shape, provider provenance, no runtime subagent credentials in artifacts | stdout/stderr, JSON output, mock provider request, event log |
|
|
45
45
|
|
|
46
46
|
## P2 Extended Confidence Batteries
|
|
@@ -49,7 +49,7 @@ entry points a user or CI runner actually executes.
|
|
|
49
49
|
| -------------------------- | ----------------------------------------------------- | ------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | ---------------------------- |
|
|
50
50
|
| Tracker and GitHub sync | Issue import/export and close readiness | opt-in CI job with network credentials | labels, comments, close gate, release readiness, no secret exposure | sanitized logs |
|
|
51
51
|
| Sonar quality loop | Local or remote Sonar import and release gate mapping | configured Sonar workflow or local compose job | insights imported, release readiness reflects quality gate, unavailable token is explicit | artifact import report |
|
|
52
|
-
| Provider-backed delegation | OpenAI, Gemini, Ollama,
|
|
52
|
+
| Provider-backed delegation | OpenAI, Anthropic, Gemini, Ollama, fake/local provider-backed routes plus runtime-native separation | opt-in provider E2E plus focused wrapper/unit coverage | registry routing, explicit direct API policy, forbidden fallback, budget and scheduler blocks, redacted evidence, no silent runtime-native fallback | redacted provider provenance |
|
|
53
53
|
| Package release dry run | npm package contents and release check | `npm pack --dry-run --json && orchestra release check --json` | generated/private state excluded, version/tag policy valid, release readiness complete | package list, release report |
|
|
54
54
|
|
|
55
55
|
## Required `/tmp` Fixture Patterns
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# Evidence Compaction
|
|
2
|
+
|
|
3
|
+
Use evidence compaction when a task has enough evidence artifacts to make
|
|
4
|
+
workflow context noisy.
|
|
5
|
+
|
|
6
|
+
```sh
|
|
7
|
+
orchestra evidence compact --task GH-471 --threshold 20
|
|
8
|
+
```
|
|
9
|
+
|
|
10
|
+
The command writes Markdown and JSON summaries under
|
|
11
|
+
`.agent-workflow/evidence-summaries/`. Raw evidence files are not modified or
|
|
12
|
+
deleted; summary artifacts keep links to every raw evidence artifact.
|
|
13
|
+
|
|
14
|
+
Compaction groups evidence by task, role, type, and result. It preserves failed
|
|
15
|
+
evidence, unresolved or residual risk lines, and acceptance criteria references
|
|
16
|
+
found through exact criterion text or `AC-<number>` mentions.
|
|
17
|
+
|
|
18
|
+
Task context rendering uses an in-memory evidence summary when the evidence
|
|
19
|
+
count reaches the configured threshold. Override the default threshold with:
|
|
20
|
+
|
|
21
|
+
```sh
|
|
22
|
+
orchestra context --task GH-471 --evidence-summary-threshold 10
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
For process-wide defaults, set `ORCHESTRA_EVIDENCE_SUMMARY_THRESHOLD`.
|
|
@@ -0,0 +1,119 @@
|
|
|
1
|
+
{
|
|
2
|
+
"generatedAt": "2026-05-22T17:09:28.420Z",
|
|
3
|
+
"workspaceRoot": "/var/folders/bv/08sjh7yj717c8shc52b5_lh00000gn/T/orchestra-context-pack-bench-YyJ38r",
|
|
4
|
+
"fixture": {
|
|
5
|
+
"requestedFiles": 120,
|
|
6
|
+
"indexedFiles": 120,
|
|
7
|
+
"excludedFiles": 2,
|
|
8
|
+
"totalIndexedBytes": 132854
|
|
9
|
+
},
|
|
10
|
+
"query": "payment command security evidence",
|
|
11
|
+
"role": "developer",
|
|
12
|
+
"phase": "developer",
|
|
13
|
+
"budget": {
|
|
14
|
+
"targetChars": 6000,
|
|
15
|
+
"hardCapChars": 8000,
|
|
16
|
+
"usedChars": 2866,
|
|
17
|
+
"maxFiles": 12,
|
|
18
|
+
"maxSnippets": 24,
|
|
19
|
+
"perSnippetChars": 600,
|
|
20
|
+
"truncated": false
|
|
21
|
+
},
|
|
22
|
+
"reduction": {
|
|
23
|
+
"sourceBytes": 132854,
|
|
24
|
+
"packedChars": 2866,
|
|
25
|
+
"ratio": 0.0216,
|
|
26
|
+
"targetSatisfied": true,
|
|
27
|
+
"hardCapSatisfied": true
|
|
28
|
+
},
|
|
29
|
+
"selectedFiles": [
|
|
30
|
+
{
|
|
31
|
+
"path": "src/payments/payment-001.ts",
|
|
32
|
+
"score": 163,
|
|
33
|
+
"inclusionReason": "path matched query; basename matched query; symbol matched query; developer phase boost",
|
|
34
|
+
"snippetCount": 1
|
|
35
|
+
},
|
|
36
|
+
{
|
|
37
|
+
"path": "src/payments/payment-007.ts",
|
|
38
|
+
"score": 163,
|
|
39
|
+
"inclusionReason": "path matched query; basename matched query; symbol matched query; developer phase boost",
|
|
40
|
+
"snippetCount": 1
|
|
41
|
+
},
|
|
42
|
+
{
|
|
43
|
+
"path": "src/payments/payment-013.ts",
|
|
44
|
+
"score": 163,
|
|
45
|
+
"inclusionReason": "path matched query; basename matched query; symbol matched query; developer phase boost",
|
|
46
|
+
"snippetCount": 1
|
|
47
|
+
},
|
|
48
|
+
{
|
|
49
|
+
"path": "src/payments/payment-019.ts",
|
|
50
|
+
"score": 163,
|
|
51
|
+
"inclusionReason": "path matched query; basename matched query; symbol matched query; developer phase boost",
|
|
52
|
+
"snippetCount": 1
|
|
53
|
+
},
|
|
54
|
+
{
|
|
55
|
+
"path": "src/payments/payment-025.ts",
|
|
56
|
+
"score": 163,
|
|
57
|
+
"inclusionReason": "path matched query; basename matched query; symbol matched query; developer phase boost",
|
|
58
|
+
"snippetCount": 1
|
|
59
|
+
},
|
|
60
|
+
{
|
|
61
|
+
"path": "src/payments/payment-031.ts",
|
|
62
|
+
"score": 163,
|
|
63
|
+
"inclusionReason": "path matched query; basename matched query; symbol matched query; developer phase boost",
|
|
64
|
+
"snippetCount": 1
|
|
65
|
+
},
|
|
66
|
+
{
|
|
67
|
+
"path": "src/payments/payment-037.ts",
|
|
68
|
+
"score": 163,
|
|
69
|
+
"inclusionReason": "path matched query; basename matched query; symbol matched query; developer phase boost",
|
|
70
|
+
"snippetCount": 1
|
|
71
|
+
},
|
|
72
|
+
{
|
|
73
|
+
"path": "src/payments/payment-043.ts",
|
|
74
|
+
"score": 163,
|
|
75
|
+
"inclusionReason": "path matched query; basename matched query; symbol matched query; developer phase boost",
|
|
76
|
+
"snippetCount": 1
|
|
77
|
+
},
|
|
78
|
+
{
|
|
79
|
+
"path": "src/payments/payment-049.ts",
|
|
80
|
+
"score": 163,
|
|
81
|
+
"inclusionReason": "path matched query; basename matched query; symbol matched query; developer phase boost",
|
|
82
|
+
"snippetCount": 1
|
|
83
|
+
},
|
|
84
|
+
{
|
|
85
|
+
"path": "src/payments/payment-055.ts",
|
|
86
|
+
"score": 163,
|
|
87
|
+
"inclusionReason": "path matched query; basename matched query; symbol matched query; developer phase boost",
|
|
88
|
+
"snippetCount": 1
|
|
89
|
+
},
|
|
90
|
+
{
|
|
91
|
+
"path": "src/payments/payment-061.ts",
|
|
92
|
+
"score": 163,
|
|
93
|
+
"inclusionReason": "path matched query; basename matched query; symbol matched query; developer phase boost",
|
|
94
|
+
"snippetCount": 1
|
|
95
|
+
},
|
|
96
|
+
{
|
|
97
|
+
"path": "src/payments/payment-067.ts",
|
|
98
|
+
"score": 163,
|
|
99
|
+
"inclusionReason": "path matched query; basename matched query; symbol matched query; developer phase boost",
|
|
100
|
+
"snippetCount": 1
|
|
101
|
+
}
|
|
102
|
+
],
|
|
103
|
+
"redactions": 12,
|
|
104
|
+
"omittedCount": 108,
|
|
105
|
+
"staleIndex": {
|
|
106
|
+
"detected": true,
|
|
107
|
+
"statusReason": "context index file changed: src/payments/payment-001.ts",
|
|
108
|
+
"packFailure": "context pack requires a fresh context index: context index file changed: src/payments/payment-001.ts"
|
|
109
|
+
},
|
|
110
|
+
"runtimeSpawnPackMetadata": {
|
|
111
|
+
"taskId": "GH-452",
|
|
112
|
+
"phase": "developer",
|
|
113
|
+
"role": "developer",
|
|
114
|
+
"jsonArtifact": ".agent-workflow/context-packs/gh-452-payment-command-security-evidence-1779469768399.json",
|
|
115
|
+
"markdownArtifact": ".agent-workflow/context-packs/gh-452-payment-command-security-evidence-1779469768399.md",
|
|
116
|
+
"contextPackArtifactPresent": true
|
|
117
|
+
},
|
|
118
|
+
"passed": true
|
|
119
|
+
}
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# Context Pack Benchmark Report
|
|
2
|
+
|
|
3
|
+
- Task: GH-452
|
|
4
|
+
- Result: passed
|
|
5
|
+
- Indexed files: 120
|
|
6
|
+
- Excluded files: 2
|
|
7
|
+
- Source bytes: 132854
|
|
8
|
+
- Packed chars: 2866
|
|
9
|
+
- Reduction ratio: 0.0216
|
|
10
|
+
- Redactions: 12
|
|
11
|
+
- Stale index detected: true
|
|
12
|
+
|
|
13
|
+
## Selected Files
|
|
14
|
+
|
|
15
|
+
- src/payments/payment-001.ts score=163 snippets=1: path matched query; basename matched query; symbol matched query; developer phase boost
|
|
16
|
+
- src/payments/payment-007.ts score=163 snippets=1: path matched query; basename matched query; symbol matched query; developer phase boost
|
|
17
|
+
- src/payments/payment-013.ts score=163 snippets=1: path matched query; basename matched query; symbol matched query; developer phase boost
|
|
18
|
+
- src/payments/payment-019.ts score=163 snippets=1: path matched query; basename matched query; symbol matched query; developer phase boost
|
|
19
|
+
- src/payments/payment-025.ts score=163 snippets=1: path matched query; basename matched query; symbol matched query; developer phase boost
|
|
20
|
+
- src/payments/payment-031.ts score=163 snippets=1: path matched query; basename matched query; symbol matched query; developer phase boost
|
|
21
|
+
- src/payments/payment-037.ts score=163 snippets=1: path matched query; basename matched query; symbol matched query; developer phase boost
|
|
22
|
+
- src/payments/payment-043.ts score=163 snippets=1: path matched query; basename matched query; symbol matched query; developer phase boost
|
|
23
|
+
- src/payments/payment-049.ts score=163 snippets=1: path matched query; basename matched query; symbol matched query; developer phase boost
|
|
24
|
+
- src/payments/payment-055.ts score=163 snippets=1: path matched query; basename matched query; symbol matched query; developer phase boost
|
|
25
|
+
- src/payments/payment-061.ts score=163 snippets=1: path matched query; basename matched query; symbol matched query; developer phase boost
|
|
26
|
+
- src/payments/payment-067.ts score=163 snippets=1: path matched query; basename matched query; symbol matched query; developer phase boost
|
|
27
|
+
|
|
28
|
+
## Runtime Spawn Pack Metadata
|
|
29
|
+
|
|
30
|
+
- JSON: .agent-workflow/context-packs/gh-452-payment-command-security-evidence-1779469768399.json
|
|
31
|
+
- Markdown: .agent-workflow/context-packs/gh-452-payment-command-security-evidence-1779469768399.md
|
|
32
|
+
- Context pack artifact present: true
|
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
# GH-428 Test Coverage And Context Review
|
|
2
|
+
|
|
3
|
+
Date: 2026-05-22
|
|
4
|
+
Task: GH-428-TEST-COVERAGE-CONTEXT-REVIEW
|
|
5
|
+
Scope: CLI, web console/site, runtime workflow gates, and context-token performance.
|
|
6
|
+
|
|
7
|
+
## Summary
|
|
8
|
+
|
|
9
|
+
Architect and QA review found no release-blocking source test gap for the current implemented behavior. The current suite has broad unit/integration coverage, browser E2E coverage for the web console and site, runtime E2E coverage for init/target guidance and queue behavior, and context-pack coverage for bounded runtime delegation.
|
|
10
|
+
|
|
11
|
+
The main remaining gaps are backlog improvements: full multi-squad E2E, workflow lifecycle CLI E2E as a standalone installed-user battery, security-boundary E2E, and stronger context budget telemetry over real-world repositories.
|
|
12
|
+
|
|
13
|
+
## Evidence Reviewed
|
|
14
|
+
|
|
15
|
+
- `npm run precommit`: passed. Remaining warnings are historical `workflow-runs.jsonl` references to archived/unknown tasks.
|
|
16
|
+
- `npm run release:matrix`: passed and lists required release flows.
|
|
17
|
+
- `npm run performance:bench`: passed with 250 tasks. `graph-plan` 118 ms, `health` 120 ms, web `/api/graph/plan` 119 ms.
|
|
18
|
+
- `node --test test/context-index.test.js test/context-search.test.js test/context-pack.test.js test/runtime-scheduler.test.js test/runtime-adapters.test.js`: 80 passing.
|
|
19
|
+
- GH-419 CI passed after push. GH-419 and prior GH-424 Sonar workflows failed at the quality-gate step after importing Orchestra evidence.
|
|
20
|
+
|
|
21
|
+
## Coverage Matrix
|
|
22
|
+
|
|
23
|
+
| Area | Current evidence | Gaps | Priority | Recommendation |
|
|
24
|
+
| --- | --- | --- | --- | --- |
|
|
25
|
+
| Source quality gate | `npm run precommit` covers lint, typecheck, secret scan, security audit, build, unit tests, workflow validation | Historical workflow-run warnings add noise | Medium | Create cleanup task for stale workflow-run references, not a release blocker |
|
|
26
|
+
| CLI onboarding | `e2e/init-onboarding.test.js` covers `/tmp` workspaces, first-use flow, target guidance refresh, acceptance mapping | Installed-package journey is listed in matrix but should be mandatory in release checklist | Medium | Promote installed-package init to release-blocking before publish |
|
|
27
|
+
| Runtime target guidance | `e2e/runtime-instruction-flow.test.js` and runtime adapter tests cover Codex, Claude, Cursor, VS Code/GitHub Copilot, Windsurf guidance contracts | Real Claude CLI validation remains user-environment dependent | Medium | Keep GH-439 for real Claude validation; add optional local smoke recipe |
|
|
28
|
+
| Runtime delegation/queue | `e2e/runtime-manual-queue.test.js`, runtime scheduler tests, parent-action tests | Multi-squad E2E listed in docs but no standalone `e2e/runtime-multi-squad.test.js` file | High | Create E2E for async multi-squad, queue promotion, independent completion, and no parent blocking |
|
|
29
|
+
| Workflow lifecycle CLI | Unit/integration workflow tests cover gates, resume, failback, no-go, handoffs | Release matrix references `e2e/workflow-lifecycle-cli.test.js`, but the file does not exist | High | Add standalone E2E for installed-user lifecycle: run, gate, request changes, failback, resume, release readiness |
|
|
30
|
+
| Web console | `e2e/web-console.spec.js` covers task creation, dashboard, providers, recovery, evidence, lifecycle, accessibility, artifact canvas | Full visual regression and screenshot review are not release-blocking by default | Medium | Add snapshot/screenshots for high-risk UI states when frontend changes |
|
|
31
|
+
| Public site | `e2e/project-site.spec.js` covers docs/site navigation and public docs behavior | Docs search/content freshness should be tied to generated docs manifest in CI | Medium | Add docs content freshness check before site build |
|
|
32
|
+
| Acceptance evidence | `test/qa-coverage.test.js`, `test/cli-output-evidence.test.js`, automation evidence tests | E2E file listed as `e2e/acceptance-evidence.test.js` does not exist | Medium | Add cross-surface acceptance evidence E2E for CLI/API/browser/deferred validation |
|
|
33
|
+
| Security boundaries | Unit tests cover web action security, secret scan, path handling | Release matrix lists `e2e/security-boundaries.test.js`, but the file does not exist | High | Add E2E for path traversal, unsafe writes, secret redaction, no raw stack traces |
|
|
34
|
+
| Context-token performance | Context index/search/pack tests and `performance:bench` pass. Context packs are bounded and fail closed on stale indexes | No large-repo telemetry baseline checked into reports | Medium | Add benchmark fixture/report for large repo context pack size, redaction count, selected files, and token budget |
|
|
35
|
+
| Sonar quality loop | Local/import contracts exist and release matrix includes Sonar | GH-419 and GH-424 Sonar workflows failed at the quality-gate step after importing Orchestra evidence | High | Treat Sonar workflow failure as operational quality-gate follow-up before release candidate promotion |
|
|
36
|
+
|
|
37
|
+
## Architect Review
|
|
38
|
+
|
|
39
|
+
The test architecture is layered correctly:
|
|
40
|
+
|
|
41
|
+
- Unit and service tests cover domain contracts, workflow gates, runtime scheduler, context pack services, and evidence rules.
|
|
42
|
+
- E2E tests run against isolated `/tmp` workspaces and validate stdout/stderr, generated files, events, and browser-visible state.
|
|
43
|
+
- Runtime context work now has a bounded context index/search/pack path, reducing full-transcript and full-file context pressure.
|
|
44
|
+
|
|
45
|
+
Architecture gaps are mostly orchestration-level:
|
|
46
|
+
|
|
47
|
+
- The release matrix names some E2E batteries that are not implemented as standalone files yet.
|
|
48
|
+
- Runtime/provider validation is intentionally mixed: deterministic source tests prove contracts, while real provider/Claude validation remains environment dependent.
|
|
49
|
+
- Sonar is part of the quality loop, but failing CI Sonar should be made visible as a release readiness blocker or accepted operational risk.
|
|
50
|
+
|
|
51
|
+
## QA Review
|
|
52
|
+
|
|
53
|
+
QA coverage is sufficient for the current GH-419/GH-424/GH-451 implemented behavior based on precommit and focused suites.
|
|
54
|
+
|
|
55
|
+
QA gaps to convert into backlog:
|
|
56
|
+
|
|
57
|
+
- Multi-squad runtime E2E must prove multiple queued/active squads, parent remains conversational, completions reconcile independently, and queued work promotes after capacity frees.
|
|
58
|
+
- Workflow lifecycle CLI E2E must prove phase handoffs, blocked transitions, return to owner, resume after correction, and release artifact content in a disposable workspace.
|
|
59
|
+
- Security boundary E2E must prove path traversal, unsafe writes, stack trace leakage, and secret-like payload redaction across CLI/API surfaces.
|
|
60
|
+
- Context performance evidence should include large-repo budgets, selected-file rationale, redaction counts, stale-index behavior, and token budget deltas.
|
|
61
|
+
|
|
62
|
+
## Follow-Up Candidates
|
|
63
|
+
|
|
64
|
+
1. High: Add standalone multi-squad runtime E2E battery: GH-455.
|
|
65
|
+
2. High: Add standalone workflow lifecycle CLI E2E battery: GH-453.
|
|
66
|
+
3. High: Add security-boundary E2E battery: GH-454.
|
|
67
|
+
4. High: Make Sonar workflow failure visible in release readiness.
|
|
68
|
+
5. Medium: Promote installed-package CLI init to release-blocking publish evidence.
|
|
69
|
+
6. Medium: Add large-repo context-pack benchmark report: GH-452.
|
|
70
|
+
7. Medium: Add docs/site content freshness check against generated docs manifest.
|
|
71
|
+
8. Medium: Add cross-surface acceptance evidence E2E.
|
|
72
|
+
|
|
73
|
+
## Release Assessment
|
|
74
|
+
|
|
75
|
+
No source implementation was performed for GH-428. Current implemented behavior is not blocked by this review. The high-priority follow-ups should be scheduled as hardening work before the next broad release candidate, especially multi-squad E2E, workflow lifecycle CLI E2E, security-boundary E2E, and Sonar release visibility.
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
# Rule Loading Strategy
|
|
2
|
+
|
|
3
|
+
Open Orchestra treats detailed delivery rules as neutral source material that can
|
|
4
|
+
be rendered or referenced by each runtime. Cursor `.mdc` files are supported
|
|
5
|
+
runtime outputs, but they are not the universal source of truth.
|
|
6
|
+
|
|
7
|
+
## Source Model
|
|
8
|
+
|
|
9
|
+
- Root files such as `AGENTS.md`, `CLAUDE.md`, and `ORCHESTRA.md` stay compact.
|
|
10
|
+
- `src/rule-catalog.ts` owns rule metadata: id, title, canonical path, roles,
|
|
11
|
+
capabilities, triggers, and risk areas.
|
|
12
|
+
- Detailed rule content lives under `rules/` using the format that best fits the
|
|
13
|
+
rule. Cursor-specific `.mdc` files remain valid rendered or legacy targets.
|
|
14
|
+
- Runtime context manifests and quality contracts resolve rules by id instead of
|
|
15
|
+
hardcoding Cursor paths.
|
|
16
|
+
|
|
17
|
+
## Runtime Behavior
|
|
18
|
+
|
|
19
|
+
For a task or phase, Orchestra selects rules from:
|
|
20
|
+
|
|
21
|
+
- active role and required roles;
|
|
22
|
+
- capabilities needed by the work;
|
|
23
|
+
- task title, goal, scope, paths, risks, and acceptance criteria;
|
|
24
|
+
- phase-specific evidence and handoff requirements.
|
|
25
|
+
|
|
26
|
+
The selected rules are injected as context references or excerpts for the active
|
|
27
|
+
runtime. A runtime may render the same rule source differently: Codex receives
|
|
28
|
+
compact markdown references, Claude can load markdown files, Cursor can receive
|
|
29
|
+
`.mdc`, and VS Code-style integrations can consume structured JSON.
|
|
30
|
+
|
|
31
|
+
## Semantic Code Rule
|
|
32
|
+
|
|
33
|
+
Implementation roles should load `semantic-code` when writing or reviewing code,
|
|
34
|
+
automation, scripts, tests, or architecture-sensitive refactors. The rule
|
|
35
|
+
requires code to be readable by intent through domain naming, narrow types,
|
|
36
|
+
focused helpers, and clear boundaries. Comments should explain why, trade-offs,
|
|
37
|
+
or non-obvious constraints, not restate the code.
|